2024-09-13T04:51:58.105 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-13T04:51:58.121 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T04:51:58.208 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} email: ceph-qa@ceph.com first_in_suite: false job_id: '7900839' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: true rbd cache max dirty: 0 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 248 sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 sleep_before_teardown: 0 subset: 116/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7a853b1706d076d58e0da7a5d3b9b9eb36276880 targets: smithi053.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKcG2CsDwrjUecfaQHjCXiPEjQvoYensJcPHU7fDgWYLerQYHD5dkg9yc9ci2NuW9n+85EZqXJnK8h75gohcN/U= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKl6daA7j4Mi0fXzl8iNkxEOCNZ9u5k6SXapUdm7NyGwKKVaHYP0L58v3v5cSyUhV7uwHko1oQQm1C1Cw4WF5jQ= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_22:16:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-13T04:51:58.208 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa; will attempt to use it 2024-09-13T04:51:58.209 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa/tasks 2024-09-13T04:51:58.209 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-13T04:51:58.210 INFO:teuthology.task.internal:Checking packages... 2024-09-13T04:51:58.231 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '48a7de7e6e1701a88fd571c7e34908ac4b8ed412' 2024-09-13T04:51:58.232 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-13T04:51:58.232 INFO:teuthology.packaging:ref: None 2024-09-13T04:51:58.232 INFO:teuthology.packaging:tag: None 2024-09-13T04:51:58.232 INFO:teuthology.packaging:branch: reef 2024-09-13T04:51:58.232 INFO:teuthology.packaging:sha1: 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:51:58.232 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-13T04:51:58.526 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-843.g2c01458a 2024-09-13T04:51:58.528 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-13T04:51:58.534 INFO:teuthology.task.internal:no buildpackages task found 2024-09-13T04:51:58.534 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-13T04:51:58.547 INFO:teuthology.task.internal:Saving configuration 2024-09-13T04:51:58.559 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-13T04:51:58.565 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-13T04:51:58.597 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi053.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 04:47:14.605880', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKcG2CsDwrjUecfaQHjCXiPEjQvoYensJcPHU7fDgWYLerQYHD5dkg9yc9ci2NuW9n+85EZqXJnK8h75gohcN/U='} 2024-09-13T04:51:58.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-13 04:47:14.604632', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKl6daA7j4Mi0fXzl8iNkxEOCNZ9u5k6SXapUdm7NyGwKKVaHYP0L58v3v5cSyUhV7uwHko1oQQm1C1Cw4WF5jQ='} 2024-09-13T04:51:58.623 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-13T04:51:58.629 INFO:teuthology.task.internal:roles: ubuntu@smithi053.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-13T04:51:58.629 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-13T04:51:58.629 INFO:teuthology.run_tasks:Running task console_log... 2024-09-13T04:51:58.761 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f242c5a85e0>, signals=[15]) 2024-09-13T04:51:58.761 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-13T04:51:58.767 INFO:teuthology.task.internal:Opening connections... 2024-09-13T04:51:58.767 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T04:51:58.769 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:51:58.870 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T04:51:58.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:51:58.972 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-13T04:51:58.979 DEBUG:teuthology.orchestra.run.smithi053:> uname -m 2024-09-13T04:51:58.997 INFO:teuthology.orchestra.run.smithi053.stdout:x86_64 2024-09-13T04:51:58.997 DEBUG:teuthology.orchestra.run.smithi053:> cat /etc/os-release 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:NAME="CentOS Stream" 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION="9" 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:ID="centos" 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:ID_LIKE="rhel fedora" 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:VERSION_ID="9" 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:PLATFORM_ID="platform:el9" 2024-09-13T04:51:59.055 INFO:teuthology.orchestra.run.smithi053.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:ANSI_COLOR="0;31" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:LOGO="fedora-logo-icon" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:HOME_URL="https://centos.org/" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T04:51:59.056 INFO:teuthology.orchestra.run.smithi053.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T04:51:59.057 INFO:teuthology.lock.ops:Updating smithi053.front.sepia.ceph.com on lock server 2024-09-13T04:51:59.084 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-09-13T04:51:59.102 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-09-13T04:51:59.102 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-09-13T04:51:59.161 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-13T04:51:59.162 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-09-13T04:51:59.162 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-13T04:51:59.162 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-13T04:51:59.162 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-13T04:51:59.162 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-09-13T04:51:59.184 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-13T04:51:59.192 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-13T04:51:59.201 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-13T04:51:59.201 DEBUG:teuthology.orchestra.run.smithi053:> test '!' -e /home/ubuntu/cephtest 2024-09-13T04:51:59.204 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-09-13T04:51:59.220 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-13T04:51:59.226 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-13T04:51:59.226 DEBUG:teuthology.orchestra.run.smithi053:> test -z $(ls -A /var/lib/ceph) 2024-09-13T04:51:59.261 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-09-13T04:51:59.287 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-13T04:51:59.353 INFO:teuthology.run_tasks:Running task kernel... 2024-09-13T04:51:59.370 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-13T04:51:59.371 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-13T04:51:59.371 DEBUG:teuthology.orchestra.run.smithi053:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T04:51:59.371 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-13T04:51:59.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:51:59.388 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-13T04:51:59.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:51:59.389 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-13T04:51:59.446 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-503.el9.x86_64 2024-09-13T04:51:59.446 INFO:teuthology.task.kernel:Running kernel on smithi053: 5.14.0-503.el9.x86_64 2024-09-13T04:51:59.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum install -y kernel 2024-09-13T04:51:59.447 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-503.el9.x86_64 2024-09-13T04:51:59.447 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-503.el9.x86_64 2024-09-13T04:51:59.448 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-09-13T04:52:02.870 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-13T04:52:03.082 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-13T04:52:05.108 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-13T04:52:06.005 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-13T04:52:09.775 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.5 MB 00:00 2024-09-13T04:52:10.356 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-13T04:52:11.268 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-13T04:52:11.981 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-13T04:52:12.164 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-13T04:52:12.927 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-13T04:52:17.870 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-13T04:52:18.417 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-13T04:52:19.968 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T04:52:19.968 INFO:teuthology.orchestra.run.smithi053.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T04:52:20.086 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T04:52:20.091 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:52:20.091 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-13T04:52:20.092 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:52:20.093 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 79 M 2024-09-13T04:52:20.093 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 126 M 2024-09-13T04:52:20.093 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T04:52:20.429 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 292 kB/s | 23 kB 00:00 2024-09-13T04:52:20.465 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-13T04:52:20.465 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-13T04:52:20.550 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T04:52:20.557 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:52:20.557 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-13T04:52:20.557 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-13T04:52:20.558 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:52:20.559 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 79 M 2024-09-13T04:52:20.559 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 126 M 2024-09-13T04:52:20.559 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T04:52:20.702 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 188 kB/s | 23 kB 00:00 2024-09-13T04:52:21.369 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-09-13T04:52:21.827 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-13T04:52:22.452 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 16 MB/s | 29 MB 00:01 2024-09-13T04:52:22.453 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:52:22.453 INFO:teuthology.orchestra.run.smithi110.stdout:Total 42 MB/s | 79 MB 00:01 2024-09-13T04:52:22.630 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.9 MB/s | 16 MB 00:02 2024-09-13T04:52:23.136 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T04:52:23.288 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 10 MB/s | 29 MB 00:02 2024-09-13T04:52:23.490 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T04:52:23.491 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T04:52:23.872 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 9.8 MB/s | 35 MB 00:03 2024-09-13T04:52:23.872 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:52:23.872 INFO:teuthology.orchestra.run.smithi053.stdout:Total 21 MB/s | 79 MB 00:03 2024-09-13T04:52:24.416 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T04:52:24.417 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T04:52:24.577 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T04:52:24.989 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T04:52:24.989 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T04:52:25.943 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T04:52:25.944 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T04:52:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T04:52:27.486 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T04:52:27.563 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T04:52:28.534 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T04:52:29.189 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-13T04:52:29.281 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T04:52:30.351 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T04:52:30.523 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T04:52:32.096 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T04:52:32.351 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T04:52:35.340 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T04:52:35.389 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:52:37.469 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T04:52:37.511 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:52:40.248 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:52:42.649 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:18.383 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:18.422 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:18.779 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:18.779 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T04:53:18.779 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T04:53:18.780 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T04:53:19.632 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:53:19.633 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T04:53:19.983 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T04:53:20.062 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T04:53:20.062 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T04:53:20.062 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T04:53:20.063 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T04:53:20.063 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T04:53:20.063 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-13T04:53:20.063 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi110.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T04:53:20.063 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T04:53:20.121 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T04:53:20.122 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T04:53:20.122 DEBUG:teuthology.orchestra.run.smithi110:> sudo rpm -qi grub2-tools 2024-09-13T04:53:20.154 INFO:teuthology.orchestra.run.smithi110.stdout:Name : grub2-tools 2024-09-13T04:53:20.154 INFO:teuthology.orchestra.run.smithi110.stdout:Epoch : 1 2024-09-13T04:53:20.154 INFO:teuthology.orchestra.run.smithi110.stdout:Version : 2.06 2024-09-13T04:53:20.154 INFO:teuthology.orchestra.run.smithi110.stdout:Release : 82.el9 2024-09-13T04:53:20.154 INFO:teuthology.orchestra.run.smithi110.stdout:Architecture: x86_64 2024-09-13T04:53:20.154 INFO:teuthology.orchestra.run.smithi110.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T04:53:20.155 INFO:teuthology.orchestra.run.smithi110.stdout:Group : Unspecified 2024-09-13T04:53:20.155 INFO:teuthology.orchestra.run.smithi110.stdout:Size : 8274670 2024-09-13T04:53:20.155 INFO:teuthology.orchestra.run.smithi110.stdout:License : GPLv3+ 2024-09-13T04:53:20.155 INFO:teuthology.orchestra.run.smithi110.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Packager : builder@centos.org 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Vendor : CentOS 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Summary : Support tools for GRUB. 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:Description : 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T04:53:20.156 INFO:teuthology.orchestra.run.smithi110.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T04:53:20.157 INFO:teuthology.orchestra.run.smithi110.stdout:hardware devices. 2024-09-13T04:53:20.157 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:53:20.157 INFO:teuthology.orchestra.run.smithi110.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T04:53:20.157 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T04:53:20.158 INFO:teuthology.task.kernel:Updating grub on smithi110 to boot 5.14.0-505.el9.x86_64 2024-09-13T04:53:20.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T04:53:20.816 INFO:teuthology.orchestra.run.smithi110.stderr:Generating grub configuration file ... 2024-09-13T04:53:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:21.967 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:22.301 INFO:teuthology.orchestra.run.smithi110.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T04:53:22.317 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-13T04:53:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-13T04:53:22.318 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-13T04:53:22.339 INFO:teuthology.orchestra.run.smithi110.stderr:done 2024-09-13T04:53:22.341 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-13T04:53:22.357 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.jfnq2dsv7O 2024-09-13T04:53:22.358 DEBUG:teuthology.orchestra.run.smithi110:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.jfnq2dsv7O 2024-09-13T04:53:22.426 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.jfnq2dsv7O 2024-09-13T04:53:22.571 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.jfnq2dsv7O is 6KB 2024-09-13T04:53:22.620 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.jfnq2dsv7O 2024-09-13T04:53:22.635 DEBUG:teuthology.orchestra.run.smithi110:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T04:53:22.706 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T04:53:22.706 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T04:53:22.706 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T04:53:22.706 INFO:teuthology.orchestra.run.smithi110.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T04:53:22.707 DEBUG:teuthology.orchestra.run.smithi110:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T04:53:22.865 DEBUG:teuthology.orchestra.run.smithi110:> sudo shutdown -r now 2024-09-13T04:53:23.175 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-13T04:53:23.175 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:53:23.175 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T04:53:23.175 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-13T04:53:23.176 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-13T04:53:23.176 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-13T04:53:23.176 INFO:teuthology.orchestra.run.smithi053.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-13T04:53:23.176 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:53:23.176 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T04:53:23.518 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T04:53:23.609 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T04:53:23.609 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T04:53:23.609 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-13T04:53:23.609 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-13T04:53:23.609 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-13T04:53:23.609 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-13T04:53:23.609 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi053.front.sepia.ceph.com, path=None, version=distro) 2024-09-13T04:53:23.610 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q kernel | sort -rV | head -n 1 2024-09-13T04:53:23.668 INFO:teuthology.orchestra.run.smithi053.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-13T04:53:23.669 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-13T04:53:23.669 DEBUG:teuthology.orchestra.run.smithi053:> sudo rpm -qi grub2-tools 2024-09-13T04:53:23.710 INFO:teuthology.orchestra.run.smithi053.stdout:Name : grub2-tools 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Epoch : 1 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Version : 2.06 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Release : 82.el9 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Architecture: x86_64 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Group : Unspecified 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Size : 8274670 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:License : GPLv3+ 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-13T04:53:23.711 INFO:teuthology.orchestra.run.smithi053.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:Packager : builder@centos.org 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:Vendor : CentOS 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:Summary : Support tools for GRUB. 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:Description : 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:hardware devices. 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:53:23.712 INFO:teuthology.orchestra.run.smithi053.stdout:This subpackage provides tools for support of all platforms. 2024-09-13T04:53:23.713 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-13T04:53:23.713 INFO:teuthology.task.kernel:Updating grub on smithi053 to boot 5.14.0-505.el9.x86_64 2024-09-13T04:53:23.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-13T04:53:24.358 INFO:teuthology.orchestra.run.smithi053.stderr:Generating grub configuration file ... 2024-09-13T04:53:25.658 INFO:teuthology.orchestra.run.smithi053.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-13T04:53:25.685 INFO:teuthology.orchestra.run.smithi053.stderr:done 2024-09-13T04:53:25.688 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-13T04:53:25.703 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.GYyy1t4yqB 2024-09-13T04:53:25.703 DEBUG:teuthology.orchestra.run.smithi053:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.GYyy1t4yqB 2024-09-13T04:53:25.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.GYyy1t4yqB 2024-09-13T04:53:25.909 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.GYyy1t4yqB is 6KB 2024-09-13T04:53:25.957 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.GYyy1t4yqB 2024-09-13T04:53:25.971 DEBUG:teuthology.orchestra.run.smithi053:> sudo /bin/ls /boot/loader/entries || true 2024-09-13T04:53:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-13T04:53:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-13T04:53:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-13T04:53:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-13T04:53:26.040 DEBUG:teuthology.orchestra.run.smithi053:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-13T04:53:26.197 DEBUG:teuthology.orchestra.run.smithi053:> sudo shutdown -r now 2024-09-13T04:53:52.895 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-13T04:53:52.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-13T04:53:52.897 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:53:56.201 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-13T04:53:56.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-13T04:53:56.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:54:52.962 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T04:54:56.214 DEBUG:teuthology.orchestra.remote:timed out 2024-09-13T04:55:01.969 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-13T04:55:01.970 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:55:02.698 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-13T04:55:03.211 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-13T04:55:03.212 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-13T04:55:03.212 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-09-13T04:55:03.259 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-505.el9.x86_64 2024-09-13T04:55:03.259 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T04:55:03.259 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T04:55:03.260 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T04:55:04.260 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-13T04:55:04.260 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T04:55:04.338 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-09-13T04:55:04.547 DEBUG:teuthology.parallel:result is None 2024-09-13T04:55:05.216 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-13T04:55:05.217 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:55:05.592 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-13T04:55:05.949 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-13T04:55:05.950 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-13T04:55:05.950 DEBUG:teuthology.orchestra.run.smithi053:> uname -r 2024-09-13T04:55:05.965 INFO:teuthology.orchestra.run.smithi053.stdout:5.14.0-505.el9.x86_64 2024-09-13T04:55:05.966 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-13T04:55:05.966 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-13T04:55:05.966 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-13T04:55:06.967 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-13T04:55:06.967 DEBUG:teuthology.orchestra.run.smithi053:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-13T04:55:07.033 INFO:teuthology.orchestra.run.smithi053.stdout:ttyS1 2024-09-13T04:55:07.057 DEBUG:teuthology.parallel:result is None 2024-09-13T04:55:07.057 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-13T04:55:07.064 INFO:teuthology.task.internal:Creating test directory... 2024-09-13T04:55:07.064 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T04:55:07.066 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-13T04:55:07.085 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-13T04:55:07.091 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-13T04:55:07.099 INFO:teuthology.task.internal:Creating archive directory... 2024-09-13T04:55:07.099 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T04:55:07.125 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-13T04:55:07.273 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-13T04:55:07.280 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-13T04:55:07.280 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T04:55:07.282 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-13T04:55:07.317 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:55:07.322 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:55:07.328 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:55:07.334 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-13T04:55:07.336 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-13T04:55:07.344 INFO:teuthology.task.internal:Configuring sudo... 2024-09-13T04:55:07.345 DEBUG:teuthology.orchestra.run.smithi053:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T04:55:07.379 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-13T04:55:07.407 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-13T04:55:07.416 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-13T04:55:07.416 DEBUG:teuthology.orchestra.run.smithi053:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T04:55:07.448 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-13T04:55:07.465 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T04:55:07.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T04:55:07.601 DEBUG:teuthology.orchestra.run.smithi053:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T04:55:07.693 DEBUG:teuthology.orchestra.run.smithi053:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T04:55:07.727 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T04:55:07.727 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T04:55:07.794 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T04:55:07.848 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-13T04:55:07.887 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T04:55:07.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-13T04:55:08.016 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T04:55:08.017 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-13T04:55:08.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo service rsyslog restart 2024-09-13T04:55:08.087 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-09-13T04:55:08.126 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T04:55:08.178 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T04:55:08.615 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-13T04:55:08.623 INFO:teuthology.task.internal:Starting timer... 2024-09-13T04:55:08.623 INFO:teuthology.run_tasks:Running task pcp... 2024-09-13T04:55:08.637 INFO:teuthology.run_tasks:Running task selinux... 2024-09-13T04:55:08.645 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-13T04:55:08.645 DEBUG:teuthology.orchestra.run.smithi053:> sudo service auditd rotate 2024-09-13T04:55:08.687 INFO:teuthology.orchestra.run.smithi053.stdout:Rotating logs: 2024-09-13T04:55:08.689 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-09-13T04:55:08.739 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-09-13T04:55:08.741 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-13T04:55:08.741 DEBUG:teuthology.orchestra.run.smithi053:> /usr/sbin/getenforce 2024-09-13T04:55:08.785 INFO:teuthology.orchestra.run.smithi053.stdout:Permissive 2024-09-13T04:55:08.786 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-09-13T04:55:08.813 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-09-13T04:55:08.813 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi053.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive'} 2024-09-13T04:55:08.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T04:55:08.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:55:08.841 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T04:55:08.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T04:55:08.882 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-13T04:55:08.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo /usr/sbin/setenforce permissive 2024-09-13T04:55:08.913 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-09-13T04:55:08.959 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-13T04:55:08.969 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-13T04:55:08.983 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-13T04:55:08.984 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi053.front.sepia.ceph.com,smithi110.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-13T04:58:54.592 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi053.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com')] 2024-09-13T04:58:54.593 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-13T04:58:54.594 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi053.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:58:54.681 DEBUG:teuthology.orchestra.run.smithi053:> true 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi053.front.sepia.ceph.com' 2024-09-13T04:58:54.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-13T04:58:54.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-13T04:58:54.852 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-09-13T04:58:54.933 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-09-13T04:58:54.933 INFO:teuthology.run_tasks:Running task clock... 2024-09-13T04:58:54.943 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-13T04:58:54.943 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T04:58:54.944 DEBUG:teuthology.orchestra.run.smithi053:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T04:58:54.947 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-13T04:58:54.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T04:58:54.978 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T04:58:54.995 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T04:58:55.003 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-13T04:58:55.019 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-13T04:58:55.023 INFO:teuthology.orchestra.run.smithi053.stderr:sudo: ntpd: command not found 2024-09-13T04:58:55.036 INFO:teuthology.orchestra.run.smithi053.stdout:506 Cannot talk to daemon 2024-09-13T04:58:55.043 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-09-13T04:58:55.051 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T04:58:55.054 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-09-13T04:58:55.066 INFO:teuthology.orchestra.run.smithi053.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T04:58:55.068 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-13T04:58:55.082 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-13T04:58:55.124 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.128 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.138 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-13T04:58:55.141 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T04:58:55.141 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-13T04:58:55.141 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.142 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.142 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.142 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T04:58:55.143 INFO:teuthology.run_tasks:Running task install... 2024-09-13T04:58:55.151 DEBUG:teuthology.task.install:project ceph 2024-09-13T04:58:55.151 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}} 2024-09-13T04:58:55.151 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'} 2024-09-13T04:58:55.151 INFO:teuthology.task.install:Using flavor: default 2024-09-13T04:58:55.157 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-13T04:58:55.157 INFO:teuthology.task.install:extra packages: [] 2024-09-13T04:58:55.157 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T04:58:55.158 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:58:55.160 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'tag': None, 'wait_for_package': False} 2024-09-13T04:58:55.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T04:58:55.314 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T04:58:55.314 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T04:58:55.334 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/ 2024-09-13T04:58:55.334 INFO:teuthology.task.install.rpm:Package version is 18.2.4-820.g48a7de7e 2024-09-13T04:58:55.442 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T04:58:55.442 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T04:58:55.442 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T04:58:55.451 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-13T04:58:55.451 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T04:58:55.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-13T04:58:55.471 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T04:58:55.471 DEBUG:teuthology.orchestra.run.smithi053:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T04:58:55.478 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-13T04:58:55.479 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-13T04:58:55.538 DEBUG:teuthology.orchestra.run.smithi053:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T04:58:55.546 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-13T04:58:55.616 DEBUG:teuthology.orchestra.run.smithi053:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T04:58:55.627 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-13T04:58:55.682 INFO:teuthology.orchestra.run.smithi053.stdout:check_obsoletes = 1 2024-09-13T04:58:55.684 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum clean all 2024-09-13T04:58:55.691 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-09-13T04:58:55.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-09-13T04:58:55.921 INFO:teuthology.orchestra.run.smithi110.stdout:45 files removed 2024-09-13T04:58:55.923 INFO:teuthology.orchestra.run.smithi053.stdout:45 files removed 2024-09-13T04:58:55.947 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw 2024-09-13T04:58:55.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-radosgw 2024-09-13T04:58:56.493 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-13T04:58:56.514 INFO:teuthology.orchestra.run.smithi053.stdout:ceph packages for x86_64 291 kB/s | 77 kB 00:00 2024-09-13T04:58:56.760 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 51 kB/s | 11 kB 00:00 2024-09-13T04:58:56.790 INFO:teuthology.orchestra.run.smithi053.stdout:ceph noarch packages 50 kB/s | 11 kB 00:00 2024-09-13T04:58:57.020 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-13T04:58:57.049 INFO:teuthology.orchestra.run.smithi053.stdout:ceph source packages 8.5 kB/s | 1.9 kB 00:00 2024-09-13T04:58:57.736 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-13T04:58:59.513 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-13T04:59:00.823 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.2 MB 00:03 2024-09-13T04:59:03.116 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-13T04:59:05.905 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.5 MB 00:02 2024-09-13T04:59:07.417 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-13T04:59:07.706 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-13T04:59:08.535 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-13T04:59:08.977 INFO:teuthology.orchestra.run.smithi053.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-13T04:59:10.014 INFO:teuthology.orchestra.run.smithi053.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-13T04:59:14.025 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-13T04:59:15.611 INFO:teuthology.orchestra.run.smithi053.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-13T04:59:16.442 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T04:59:16.455 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T04:59:16.456 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T04:59:16.457 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout:Install 28 Packages 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 2 Packages 2024-09-13T04:59:16.458 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 53 M 2024-09-13T04:59:16.459 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T04:59:17.318 INFO:teuthology.orchestra.run.smithi110.stdout:(1/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 10 MB/s | 5.1 MB 00:00 2024-09-13T04:59:17.368 INFO:teuthology.orchestra.run.smithi110.stdout:(2/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 504 kB/s | 25 kB 00:00 2024-09-13T04:59:17.460 INFO:teuthology.orchestra.run.smithi110.stdout:(3/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 12 MB/s | 7.7 MB 00:00 2024-09-13T04:59:17.502 INFO:teuthology.orchestra.run.smithi110.stdout:(4/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 5.2 MB/s | 711 kB 00:00 2024-09-13T04:59:17.528 INFO:teuthology.orchestra.run.smithi110.stdout:(5/30): libradosstriper1-18.2.4-820.g48a7de7e.e 7.0 MB/s | 478 kB 00:00 2024-09-13T04:59:17.578 INFO:teuthology.orchestra.run.smithi110.stdout:(6/30): python3-ceph-argparse-18.2.4-820.g48a7d 900 kB/s | 45 kB 00:00 2024-09-13T04:59:17.645 INFO:teuthology.orchestra.run.smithi110.stdout:(7/30): python3-ceph-common-18.2.4-820.g48a7de7 1.9 MB/s | 129 kB 00:00 2024-09-13T04:59:17.711 INFO:teuthology.orchestra.run.smithi110.stdout:(8/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 2.4 MB/s | 162 kB 00:00 2024-09-13T04:59:17.778 INFO:teuthology.orchestra.run.smithi110.stdout:(9/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_64 16 MB/s | 4.5 MB 00:00 2024-09-13T04:59:17.804 INFO:teuthology.orchestra.run.smithi110.stdout:(10/30): python3-rados-18.2.4-820.g48a7de7e.el9 3.4 MB/s | 321 kB 00:00 2024-09-13T04:59:17.971 INFO:teuthology.orchestra.run.smithi110.stdout:(11/30): ceph-common-18.2.4-820.g48a7de7e.el9.x 16 MB/s | 18 MB 00:01 2024-09-13T04:59:17.996 INFO:teuthology.orchestra.run.smithi110.stdout:(12/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 517 kB/s | 99 kB 00:00 2024-09-13T04:59:18.030 INFO:teuthology.orchestra.run.smithi110.stdout:(13/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 1.2 MB/s | 299 kB 00:00 2024-09-13T04:59:18.081 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 7.7 M 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrading: 2024-09-13T04:59:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: librados2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.3 M 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 5.1 M 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 18 M 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 25 k 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 711 k 2024-09-13T04:59:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 478 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2 x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.5 M 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 45 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 129 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 162 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-13T04:59:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 321 k 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 299 k 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 99 k 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout:Install 28 Packages 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout:Upgrade 2 Packages 2024-09-13T04:59:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:18.099 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 53 M 2024-09-13T04:59:18.099 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T04:59:18.397 INFO:teuthology.orchestra.run.smithi110.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 123 kB/s | 45 kB 00:00 2024-09-13T04:59:18.455 INFO:teuthology.orchestra.run.smithi110.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 221 kB/s | 107 kB 00:00 2024-09-13T04:59:18.589 INFO:teuthology.orchestra.run.smithi110.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 270 kB/s | 160 kB 00:00 2024-09-13T04:59:18.671 INFO:teuthology.orchestra.run.smithi053.stdout:(1/30): ceph-base-18.2.4-820.g48a7de7e.el9.x86_ 10 MB/s | 5.1 MB 00:00 2024-09-13T04:59:18.698 INFO:teuthology.orchestra.run.smithi110.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 390 kB/s | 42 kB 00:00 2024-09-13T04:59:18.729 INFO:teuthology.orchestra.run.smithi053.stdout:(2/30): ceph-selinux-18.2.4-820.g48a7de7e.el9.x 433 kB/s | 25 kB 00:00 2024-09-13T04:59:18.748 INFO:teuthology.orchestra.run.smithi110.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-09-13T04:59:18.781 INFO:teuthology.orchestra.run.smithi110.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 1.3 MB/s | 111 kB 00:00 2024-09-13T04:59:18.823 INFO:teuthology.orchestra.run.smithi110.stdout:(20/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 4.0 MB/s | 308 kB 00:00 2024-09-13T04:59:18.840 INFO:teuthology.orchestra.run.smithi110.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-13T04:59:18.855 INFO:teuthology.orchestra.run.smithi053.stdout:(3/30): ceph-radosgw-18.2.4-820.g48a7de7e.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-13T04:59:18.859 INFO:teuthology.orchestra.run.smithi110.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-13T04:59:18.883 INFO:teuthology.orchestra.run.smithi110.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-13T04:59:18.888 INFO:teuthology.orchestra.run.smithi053.stdout:(4/30): libcephfs2-18.2.4-820.g48a7de7e.el9.x86 4.4 MB/s | 711 kB 00:00 2024-09-13T04:59:18.914 INFO:teuthology.orchestra.run.smithi053.stdout:(5/30): libradosstriper1-18.2.4-820.g48a7de7e.e 8.0 MB/s | 478 kB 00:00 2024-09-13T04:59:18.950 INFO:teuthology.orchestra.run.smithi110.stdout:(24/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-13T04:59:18.964 INFO:teuthology.orchestra.run.smithi053.stdout:(6/30): python3-ceph-argparse-18.2.4-820.g48a7d 904 kB/s | 45 kB 00:00 2024-09-13T04:59:18.983 INFO:teuthology.orchestra.run.smithi110.stdout:(25/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-09-13T04:59:19.017 INFO:teuthology.orchestra.run.smithi110.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-13T04:59:19.031 INFO:teuthology.orchestra.run.smithi053.stdout:(7/30): python3-ceph-common-18.2.4-820.g48a7de7 1.9 MB/s | 129 kB 00:00 2024-09-13T04:59:19.084 INFO:teuthology.orchestra.run.smithi110.stdout:(27/30): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-13T04:59:19.134 INFO:teuthology.orchestra.run.smithi110.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-13T04:59:19.239 INFO:teuthology.orchestra.run.smithi053.stdout:(8/30): ceph-common-18.2.4-820.g48a7de7e.el9.x8 17 MB/s | 18 MB 00:01 2024-09-13T04:59:19.265 INFO:teuthology.orchestra.run.smithi053.stdout:(9/30): python3-cephfs-18.2.4-820.g48a7de7e.el9 692 kB/s | 162 kB 00:00 2024-09-13T04:59:19.307 INFO:teuthology.orchestra.run.smithi053.stdout:(10/30): python3-rados-18.2.4-820.g48a7de7e.el9 4.7 MB/s | 321 kB 00:00 2024-09-13T04:59:19.309 INFO:teuthology.orchestra.run.smithi110.stdout:(29/30): librados2-18.2.4-820.g48a7de7e.el9.x86 11 MB/s | 3.3 MB 00:00 2024-09-13T04:59:19.368 INFO:teuthology.orchestra.run.smithi110.stdout:(30/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 11 MB/s | 3.0 MB 00:00 2024-09-13T04:59:19.374 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:59:19.374 INFO:teuthology.orchestra.run.smithi110.stdout:Total 18 MB/s | 53 MB 00:02 2024-09-13T04:59:19.382 INFO:teuthology.orchestra.run.smithi053.stdout:(11/30): librgw2-18.2.4-820.g48a7de7e.el9.x86_6 9.0 MB/s | 4.5 MB 00:00 2024-09-13T04:59:19.407 INFO:teuthology.orchestra.run.smithi053.stdout:(12/30): python3-rgw-18.2.4-820.g48a7de7e.el9.x 991 kB/s | 99 kB 00:00 2024-09-13T04:59:19.433 INFO:teuthology.orchestra.run.smithi053.stdout:(13/30): python3-rbd-18.2.4-820.g48a7de7e.el9.x 1.7 MB/s | 299 kB 00:00 2024-09-13T04:59:19.448 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T04:59:19.483 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T04:59:19.484 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T04:59:19.541 INFO:teuthology.orchestra.run.smithi053.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 673 kB/s | 107 kB 00:00 2024-09-13T04:59:19.566 INFO:teuthology.orchestra.run.smithi053.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-13T04:59:19.600 INFO:teuthology.orchestra.run.smithi053.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 831 kB/s | 160 kB 00:00 2024-09-13T04:59:19.650 INFO:teuthology.orchestra.run.smithi053.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 847 kB/s | 42 kB 00:00 2024-09-13T04:59:19.693 INFO:teuthology.orchestra.run.smithi053.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-13T04:59:19.717 INFO:teuthology.orchestra.run.smithi053.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-13T04:59:19.751 INFO:teuthology.orchestra.run.smithi053.stdout:(20/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 5.1 MB/s | 308 kB 00:00 2024-09-13T04:59:19.776 INFO:teuthology.orchestra.run.smithi053.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-09-13T04:59:19.802 INFO:teuthology.orchestra.run.smithi053.stdout:(22/30): librdkafka-1.6.1-102.el9.x86_64.rpm 2.5 MB/s | 662 kB 00:00 2024-09-13T04:59:19.827 INFO:teuthology.orchestra.run.smithi053.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-13T04:59:19.877 INFO:teuthology.orchestra.run.smithi053.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 898 kB/s | 67 kB 00:00 2024-09-13T04:59:19.919 INFO:teuthology.orchestra.run.smithi053.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 839 kB 00:00 2024-09-13T04:59:19.945 INFO:teuthology.orchestra.run.smithi053.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-13T04:59:20.028 INFO:teuthology.orchestra.run.smithi053.stdout:(27/30): libarrow-9.0.0-11.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-09-13T04:59:20.078 INFO:teuthology.orchestra.run.smithi053.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-13T04:59:20.132 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T04:59:20.132 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T04:59:20.187 INFO:teuthology.orchestra.run.smithi053.stdout:(29/30): librados2-18.2.4-820.g48a7de7e.el9.x86 14 MB/s | 3.3 MB 00:00 2024-09-13T04:59:20.254 INFO:teuthology.orchestra.run.smithi053.stdout:(30/30): librbd1-18.2.4-820.g48a7de7e.el9.x86_6 13 MB/s | 3.0 MB 00:00 2024-09-13T04:59:20.258 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:59:20.258 INFO:teuthology.orchestra.run.smithi053.stdout:Total 25 MB/s | 53 MB 00:02 2024-09-13T04:59:20.330 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T04:59:20.368 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T04:59:20.368 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T04:59:20.992 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T04:59:20.992 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T04:59:21.524 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T04:59:21.617 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T04:59:21.649 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T04:59:21.698 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T04:59:22.070 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T04:59:22.122 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T04:59:22.197 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T04:59:22.282 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T04:59:22.316 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T04:59:22.362 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T04:59:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T04:59:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T04:59:22.508 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T04:59:22.534 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T04:59:22.535 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T04:59:22.601 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T04:59:22.644 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-13T04:59:22.684 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-13T04:59:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T04:59:22.742 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T04:59:22.743 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-13T04:59:22.802 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T04:59:22.832 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T04:59:22.859 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T04:59:22.895 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T04:59:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T04:59:23.113 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-13T04:59:23.199 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T04:59:23.243 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T04:59:23.283 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T04:59:23.293 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T04:59:23.356 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T04:59:23.379 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T04:59:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T04:59:23.409 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T04:59:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T04:59:23.464 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T04:59:23.501 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-13T04:59:23.545 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 9/32 2024-09-13T04:59:23.613 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T04:59:23.636 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T04:59:23.685 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T04:59:23.795 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-13T04:59:23.826 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T04:59:23.827 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-13T04:59:23.852 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T04:59:23.868 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-13T04:59:23.909 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-13T04:59:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T04:59:23.937 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-13T04:59:23.980 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-13T04:59:23.999 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T04:59:24.022 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-13T04:59:24.030 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T04:59:24.336 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-13T04:59:24.376 INFO:teuthology.orchestra.run.smithi053.stdout: Upgrading : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T04:59:24.450 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 20/32 2024-09-13T04:59:24.481 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 21/32 2024-09-13T04:59:24.525 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-13T04:59:24.957 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-13T04:59:24.978 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T04:59:25.050 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 24/32 2024-09-13T04:59:25.130 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 25/32 2024-09-13T04:59:25.164 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 26/32 2024-09-13T04:59:26.131 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T04:59:26.177 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T04:59:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T04:59:26.747 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T04:59:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T04:59:26.831 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T04:59:26.831 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T04:59:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T04:59:27.359 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T04:59:27.510 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T04:59:28.095 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 27/32 2024-09-13T04:59:28.138 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T04:59:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 28/32 2024-09-13T04:59:28.243 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-13T04:59:28.243 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-13T04:59:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:28.314 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T04:59:36.787 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T04:59:36.787 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-09-13T04:59:36.787 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-09-13T04:59:36.787 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-09-13T04:59:36.787 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-09-13T04:59:36.788 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-09-13T04:59:36.788 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-09-13T04:59:36.788 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-09-13T04:59:36.788 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:37.536 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T04:59:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T04:59:37.561 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:37.561 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T04:59:37.561 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:59:37.562 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:59:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T04:59:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T04:59:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T04:59:38.797 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T04:59:38.797 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T04:59:38.797 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T04:59:38.797 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T04:59:38.797 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T04:59:38.798 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T04:59:38.799 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T04:59:38.800 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /sys 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /proc 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /mnt 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /var/tmp 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /home 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /root 2024-09-13T04:59:38.922 INFO:teuthology.orchestra.run.smithi053.stdout:skipping the directory /tmp 2024-09-13T04:59:38.923 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:39.694 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T04:59:39.720 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 30/32 2024-09-13T04:59:39.720 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:39.720 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-13T04:59:39.720 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:59:39.720 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-13T04:59:39.721 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:39.725 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T04:59:39.813 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-13T04:59:39.813 INFO:teuthology.orchestra.run.smithi053.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.916 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.917 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:39.918 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T04:59:39.919 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T04:59:39.919 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:39.919 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T04:59:40.357 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-test 2024-09-13T04:59:40.919 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 1/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 3/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 4/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 5/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 6/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 7/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x 8/32 2024-09-13T04:59:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86 9/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 10/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 11/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 12/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 13/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-13T04:59:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-13T04:59:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 29/32 2024-09-13T04:59:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-13T04:59:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 31/32 2024-09-13T04:59:41.063 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T04:59:41.063 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:41.063 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T04:59:41.063 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:41.063 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T04:59:41.063 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-09-13T04:59:41.064 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:41.065 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 40 M 2024-09-13T04:59:41.065 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 164 M 2024-09-13T04:59:41.065 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T04:59:41.671 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-13T04:59:41.696 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 709 kB/s | 166 kB 00:00 2024-09-13T04:59:41.721 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout:Upgraded: 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout: librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout: librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-13T04:59:41.962 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-radosgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-selinux-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: libradosstriper1-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.963 INFO:teuthology.orchestra.run.smithi053.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: librgw2-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-argparse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ceph-common-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: re2-1:20211101-20.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-13T04:59:41.964 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:41.965 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T04:59:42.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-test 2024-09-13T04:59:42.788 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 30 MB/s | 40 MB 00:01 2024-09-13T04:59:42.790 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:59:42.791 INFO:teuthology.orchestra.run.smithi110.stdout:Total 23 MB/s | 40 MB 00:01 2024-09-13T04:59:42.819 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T04:59:42.839 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T04:59:42.839 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T04:59:42.908 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:27 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T04:59:43.046 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 40 M 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T04:59:43.047 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 166 k 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout:Install 4 Packages 2024-09-13T04:59:43.048 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:43.049 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 40 M 2024-09-13T04:59:43.049 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 164 M 2024-09-13T04:59:43.049 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T04:59:43.163 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T04:59:43.163 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T04:59:43.512 INFO:teuthology.orchestra.run.smithi053.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.4 MB/s | 304 kB 00:00 2024-09-13T04:59:43.536 INFO:teuthology.orchestra.run.smithi053.stdout:(2/4): libcephsqlite-18.2.4-820.g48a7de7e.el9.x 709 kB/s | 166 kB 00:00 2024-09-13T04:59:43.578 INFO:teuthology.orchestra.run.smithi053.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 958 kB/s | 64 kB 00:00 2024-09-13T04:59:43.635 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T04:59:43.724 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T04:59:43.763 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T04:59:43.790 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T04:59:44.587 INFO:teuthology.orchestra.run.smithi053.stdout:(4/4): ceph-test-18.2.4-820.g48a7de7e.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-13T04:59:44.589 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:59:44.589 INFO:teuthology.orchestra.run.smithi053.stdout:Total 26 MB/s | 40 MB 00:01 2024-09-13T04:59:44.618 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T04:59:44.640 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T04:59:44.640 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T04:59:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T04:59:44.980 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T04:59:45.481 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T04:59:45.565 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-13T04:59:45.603 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-13T04:59:45.647 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T04:59:47.440 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T04:59:47.523 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T04:59:49.588 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 3/4 2024-09-13T04:59:49.673 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T04:59:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T04:59:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T04:59:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T04:59:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T04:59:49.984 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:49.985 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T04:59:50.295 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph 2024-09-13T04:59:50.853 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:37 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T04:59:51.001 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T04:59:51.004 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:51.004 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T04:59:51.004 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:51.004 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T04:59:51.004 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T04:59:51.004 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T04:59:51.005 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T04:59:51.006 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout:Install 33 Packages 2024-09-13T04:59:51.007 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:51.008 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 29 M 2024-09-13T04:59:51.008 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 104 M 2024-09-13T04:59:51.008 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T04:59:51.490 INFO:teuthology.orchestra.run.smithi110.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 49 kB/s | 6.5 kB 00:00 2024-09-13T04:59:51.665 INFO:teuthology.orchestra.run.smithi110.stdout:(2/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-13T04:59:51.682 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 4/4 2024-09-13T04:59:51.682 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 1/4 2024-09-13T04:59:51.682 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2/4 2024-09-13T04:59:51.682 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-13T04:59:51.707 INFO:teuthology.orchestra.run.smithi110.stdout:(3/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 6.0 MB/s | 2.1 MB 00:00 2024-09-13T04:59:51.816 INFO:teuthology.orchestra.run.smithi110.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.2 MB/s | 246 kB 00:00 2024-09-13T04:59:51.891 INFO:teuthology.orchestra.run.smithi110.stdout:(5/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-13T04:59:52.000 INFO:teuthology.orchestra.run.smithi110.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-13T04:59:52.019 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-test-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout: libcephsqlite-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:52.020 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T04:59:52.042 INFO:teuthology.orchestra.run.smithi110.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-13T04:59:52.075 INFO:teuthology.orchestra.run.smithi110.stdout:(8/33): python3-cryptography-36.0.1-4.el9.x86_6 6.7 MB/s | 1.2 MB 00:00 2024-09-13T04:59:52.101 INFO:teuthology.orchestra.run.smithi110.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-09-13T04:59:52.134 INFO:teuthology.orchestra.run.smithi110.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.1 MB/s | 125 kB 00:00 2024-09-13T04:59:52.159 INFO:teuthology.orchestra.run.smithi110.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-13T04:59:52.335 INFO:teuthology.orchestra.run.smithi110.stdout:(12/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_ 25 MB/s | 17 MB 00:00 2024-09-13T04:59:52.394 INFO:teuthology.orchestra.run.smithi110.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 665 kB/s | 172 kB 00:00 2024-09-13T04:59:52.419 INFO:teuthology.orchestra.run.smithi110.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 134 kB/s | 35 kB 00:00 2024-09-13T04:59:52.452 INFO:teuthology.orchestra.run.smithi110.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-13T04:59:52.465 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph 2024-09-13T04:59:52.478 INFO:teuthology.orchestra.run.smithi110.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 495 kB/s | 42 kB 00:00 2024-09-13T04:59:52.511 INFO:teuthology.orchestra.run.smithi110.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 471 kB/s | 43 kB 00:00 2024-09-13T04:59:52.545 INFO:teuthology.orchestra.run.smithi110.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 425 kB/s | 11 kB 00:00 2024-09-13T04:59:52.570 INFO:teuthology.orchestra.run.smithi110.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-13T04:59:52.587 INFO:teuthology.orchestra.run.smithi110.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 438 kB/s | 18 kB 00:00 2024-09-13T04:59:52.604 INFO:teuthology.orchestra.run.smithi110.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 683 kB/s | 23 kB 00:00 2024-09-13T04:59:52.621 INFO:teuthology.orchestra.run.smithi110.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 576 kB/s | 19 kB 00:00 2024-09-13T04:59:52.638 INFO:teuthology.orchestra.run.smithi110.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 588 kB/s | 20 kB 00:00 2024-09-13T04:59:52.655 INFO:teuthology.orchestra.run.smithi110.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 1.3 MB/s | 46 kB 00:00 2024-09-13T04:59:52.681 INFO:teuthology.orchestra.run.smithi110.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.8 MB/s | 79 kB 00:00 2024-09-13T04:59:52.706 INFO:teuthology.orchestra.run.smithi110.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.6 MB/s | 358 kB 00:00 2024-09-13T04:59:52.723 INFO:teuthology.orchestra.run.smithi110.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 389 kB/s | 16 kB 00:00 2024-09-13T04:59:52.748 INFO:teuthology.orchestra.run.smithi110.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-13T04:59:52.773 INFO:teuthology.orchestra.run.smithi110.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 713 kB/s | 36 kB 00:00 2024-09-13T04:59:52.799 INFO:teuthology.orchestra.run.smithi110.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-13T04:59:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 546 kB/s | 20 kB 00:00 2024-09-13T04:59:52.861 INFO:teuthology.orchestra.run.smithi110.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.0 MB/s | 230 kB 00:00 2024-09-13T04:59:52.886 INFO:teuthology.orchestra.run.smithi110.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.7 MB/s | 427 kB 00:00 2024-09-13T04:59:52.890 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:59:52.890 INFO:teuthology.orchestra.run.smithi110.stdout:Total 15 MB/s | 29 MB 00:01 2024-09-13T04:59:53.006 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T04:59:53.043 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T04:59:53.043 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T04:59:53.053 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:38 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T04:59:53.195 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T04:59:53.197 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: ceph x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 6.5 k 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 2.1 M 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 1.5 M 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 246 k 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 4.7 M 2024-09-13T04:59:53.198 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 17 M 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-13T04:59:53.199 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-13T04:59:53.200 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout:Install 33 Packages 2024-09-13T04:59:53.201 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T04:59:53.202 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 29 M 2024-09-13T04:59:53.202 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 104 M 2024-09-13T04:59:53.202 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T04:59:53.445 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T04:59:53.445 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T04:59:53.554 INFO:teuthology.orchestra.run.smithi053.stdout:(1/33): ceph-18.2.4-820.g48a7de7e.el9.x86_64.rp 51 kB/s | 6.5 kB 00:00 2024-09-13T04:59:53.762 INFO:teuthology.orchestra.run.smithi053.stdout:(2/33): ceph-mgr-18.2.4-820.g48a7de7e.el9.x86_6 4.3 MB/s | 1.5 MB 00:00 2024-09-13T04:59:53.812 INFO:teuthology.orchestra.run.smithi053.stdout:(3/33): ceph-mds-18.2.4-820.g48a7de7e.el9.x86_6 5.4 MB/s | 2.1 MB 00:00 2024-09-13T04:59:53.896 INFO:teuthology.orchestra.run.smithi053.stdout:(4/33): ceph-mgr-modules-core-18.2.4-820.g48a7d 2.9 MB/s | 246 kB 00:00 2024-09-13T04:59:54.005 INFO:teuthology.orchestra.run.smithi053.stdout:(5/33): ceph-mon-18.2.4-820.g48a7de7e.el9.x86_6 10 MB/s | 4.7 MB 00:00 2024-09-13T04:59:54.113 INFO:teuthology.orchestra.run.smithi053.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-13T04:59:54.167 INFO:teuthology.orchestra.run.smithi053.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.0 MB/s | 106 kB 00:00 2024-09-13T04:59:54.343 INFO:teuthology.orchestra.run.smithi053.stdout:(8/33): ceph-osd-18.2.4-820.g48a7de7e.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-13T04:59:54.368 INFO:teuthology.orchestra.run.smithi053.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 675 kB/s | 135 kB 00:00 2024-09-13T04:59:54.459 INFO:teuthology.orchestra.run.smithi053.stdout:(10/33): python3-urllib3-1.26.5-6.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-09-13T04:59:54.543 INFO:teuthology.orchestra.run.smithi053.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 2.3 MB/s | 1.2 MB 00:00 2024-09-13T04:59:54.569 INFO:teuthology.orchestra.run.smithi053.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 553 kB/s | 125 kB 00:00 2024-09-13T04:59:54.602 INFO:teuthology.orchestra.run.smithi053.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-09-13T04:59:54.627 INFO:teuthology.orchestra.run.smithi053.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 415 kB/s | 35 kB 00:00 2024-09-13T04:59:54.653 INFO:teuthology.orchestra.run.smithi053.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-13T04:59:54.678 INFO:teuthology.orchestra.run.smithi053.stdout:(16/33): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-13T04:59:54.702 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T04:59:54.728 INFO:teuthology.orchestra.run.smithi053.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 432 kB/s | 43 kB 00:00 2024-09-13T04:59:54.753 INFO:teuthology.orchestra.run.smithi053.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 468 kB/s | 11 kB 00:00 2024-09-13T04:59:54.765 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T04:59:54.779 INFO:teuthology.orchestra.run.smithi053.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-13T04:59:54.797 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T04:59:54.804 INFO:teuthology.orchestra.run.smithi053.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-13T04:59:54.829 INFO:teuthology.orchestra.run.smithi053.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-13T04:59:54.830 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T04:59:54.854 INFO:teuthology.orchestra.run.smithi053.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-13T04:59:54.880 INFO:teuthology.orchestra.run.smithi053.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-13T04:59:54.905 INFO:teuthology.orchestra.run.smithi053.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-13T04:59:54.920 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T04:59:54.930 INFO:teuthology.orchestra.run.smithi053.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-13T04:59:54.956 INFO:teuthology.orchestra.run.smithi053.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-09-13T04:59:54.981 INFO:teuthology.orchestra.run.smithi053.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-13T04:59:55.006 INFO:teuthology.orchestra.run.smithi053.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-13T04:59:55.007 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T04:59:55.031 INFO:teuthology.orchestra.run.smithi053.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-13T04:59:55.049 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T04:59:55.057 INFO:teuthology.orchestra.run.smithi053.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-13T04:59:55.082 INFO:teuthology.orchestra.run.smithi053.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-09-13T04:59:55.084 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T04:59:55.107 INFO:teuthology.orchestra.run.smithi053.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-13T04:59:55.123 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T04:59:55.132 INFO:teuthology.orchestra.run.smithi053.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-13T04:59:55.137 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T04:59:55.137 INFO:teuthology.orchestra.run.smithi053.stdout:Total 15 MB/s | 29 MB 00:01 2024-09-13T04:59:55.165 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T04:59:55.208 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T04:59:55.248 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T04:59:55.263 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T04:59:55.283 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T04:59:55.305 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T04:59:55.305 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T04:59:55.317 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T04:59:55.366 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T04:59:55.454 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T04:59:55.591 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T04:59:55.673 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T04:59:55.733 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T04:59:55.741 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T04:59:55.741 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T04:59:55.773 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T04:59:55.822 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T04:59:55.877 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T04:59:56.070 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T04:59:56.133 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T04:59:56.213 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T04:59:56.318 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T04:59:56.368 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T04:59:56.493 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T04:59:56.661 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T04:59:56.693 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T04:59:56.720 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T04:59:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:56.720 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T04:59:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T04:59:56.721 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T04:59:56.721 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:57.056 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T04:59:57.127 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-13T04:59:57.168 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-13T04:59:57.210 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-13T04:59:57.304 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-13T04:59:57.398 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-13T04:59:57.445 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-13T04:59:57.489 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-13T04:59:57.518 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-13T04:59:57.553 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-13T04:59:57.586 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-13T04:59:57.627 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-13T04:59:57.670 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-13T04:59:57.713 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-13T04:59:57.767 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-13T04:59:57.851 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-13T04:59:57.994 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-13T04:59:58.448 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T04:59:58.449 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T04:59:58.450 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-13T04:59:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-13T04:59:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-13T04:59:58.450 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-13T04:59:58.451 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-13T04:59:58.499 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-13T04:59:58.546 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-13T04:59:58.628 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T04:59:58.643 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-13T04:59:58.652 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T04:59:58.652 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:58.652 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T04:59:58.652 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T04:59:58.653 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T04:59:58.653 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:58.733 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-13T04:59:58.785 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-13T04:59:58.862 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T04:59:58.886 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T04:59:58.886 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:58.886 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T04:59:58.886 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T04:59:58.886 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T04:59:58.887 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T04:59:58.915 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-13T04:59:58.951 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T04:59:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 28/33 2024-09-13T04:59:59.121 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T04:59:59.147 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 29/33 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-13T04:59:59.148 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:00.655 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T05:00:00.698 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 30/33 2024-09-13T05:00:00.698 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:00.698 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-13T05:00:00.698 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T05:00:00.698 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-13T05:00:00.698 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:00.770 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T05:00:00.770 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T05:00:00.771 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T05:00:00.772 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T05:00:00.773 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T05:00:00.773 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T05:00:00.773 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T05:00:00.773 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T05:00:00.774 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T05:00:00.774 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T05:00:00.774 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T05:00:01.173 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T05:00:01.199 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 31/33 2024-09-13T05:00:01.199 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:01.199 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-13T05:00:01.199 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T05:00:01.199 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-13T05:00:01.199 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:01.415 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T05:00:01.441 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 32/33 2024-09-13T05:00:01.441 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:01.441 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-13T05:00:01.441 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T05:00:01.441 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-13T05:00:01.441 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:01.506 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T05:00:01.830 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T05:00:01.830 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T05:00:01.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T05:00:01.832 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:01.833 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:02.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-base 2024-09-13T05:00:02.788 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:49 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:02.877 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:02.919 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:02.920 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:02.920 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:03.007 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install cephadm 2024-09-13T05:00:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 33/33 2024-09-13T05:00:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 1/33 2024-09-13T05:00:03.429 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 3/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 4/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 5/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.n 6/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-13T05:00:03.430 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-13T05:00:03.431 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-13T05:00:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-13T05:00:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-13T05:00:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-13T05:00:03.432 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-13T05:00:03.564 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:03.693 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:03.694 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:03.695 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:03.695 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:00:03.695 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:03.695 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 226 k 2024-09-13T05:00:03.695 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 217 k 2024-09-13T05:00:03.695 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:03.904 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 1.1 MB/s | 226 kB 00:00 2024-09-13T05:00:03.904 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:03.904 INFO:teuthology.orchestra.run.smithi110.stdout:Total 1.0 MB/s | 226 kB 00:00 2024-09-13T05:00:03.904 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:03.909 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:03.909 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:03.912 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:03.913 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:03.981 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:04.366 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:04.393 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mds-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-modules-core-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mon-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-osd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-13T05:00:04.394 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-ply-3.11-14.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-13T05:00:04.395 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:04.396 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:04.449 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:04.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-base 2024-09-13T05:00:04.848 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:05.096 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:05.096 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:05.313 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:50 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:05.316 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T05:00:05.406 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-base-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:05.450 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:05.451 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:05.451 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:05.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install cephadm 2024-09-13T05:00:05.889 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:52 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:06.021 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 143 k 2024-09-13T05:00:06.022 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 395 k 2024-09-13T05:00:06.023 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:06.105 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:51 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:06.203 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 794 kB/s | 143 kB 00:00 2024-09-13T05:00:06.203 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:06.204 INFO:teuthology.orchestra.run.smithi110.stdout:Total 787 kB/s | 143 kB 00:00 2024-09-13T05:00:06.204 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:06.211 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:06.211 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:06.243 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:06.244 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:06.244 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:06.244 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:06.244 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:06.244 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 226 k 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 226 k 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 217 k 2024-09-13T05:00:06.245 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:06.278 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:06.278 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:06.420 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:06.480 INFO:teuthology.orchestra.run.smithi053.stdout:cephadm-18.2.4-820.g48a7de7e.el9.noarch.rpm 961 kB/s | 226 kB 00:00 2024-09-13T05:00:06.481 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:06.481 INFO:teuthology.orchestra.run.smithi053.stdout:Total 955 kB/s | 226 kB 00:00 2024-09-13T05:00:06.481 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:06.487 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:06.487 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:06.490 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:06.490 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:06.503 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T05:00:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T05:00:06.529 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:06.529 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T05:00:06.529 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:06.560 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:07.020 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:07.114 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T05:00:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:07.429 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:07.429 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:07.429 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:07.511 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:07.699 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr 2024-09-13T05:00:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/1 2024-09-13T05:00:07.787 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:07.788 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:07.788 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:07.788 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:07.995 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-immutable-object-cache 2024-09-13T05:00:08.278 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:08.368 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:08.409 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:08.410 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:08.410 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:08.498 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T05:00:08.587 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:53 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:08.725 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 143 k 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:08.726 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:00:08.727 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:08.727 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 143 k 2024-09-13T05:00:08.727 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 395 k 2024-09-13T05:00:08.727 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:08.913 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-immutable-object-cache-18.2.4-820.g48a7de7 767 kB/s | 143 kB 00:00 2024-09-13T05:00:08.913 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:08.914 INFO:teuthology.orchestra.run.smithi053.stdout:Total 760 kB/s | 143 kB 00:00 2024-09-13T05:00:08.914 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:08.922 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:08.922 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:08.983 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:08.983 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:09.054 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:09.136 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:09.185 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:09.185 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:09.186 INFO:teuthology.orchestra.run.smithi110.stdout:Install 5 Packages 2024-09-13T05:00:09.187 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:09.187 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 3.8 M 2024-09-13T05:00:09.187 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 87 M 2024-09-13T05:00:09.187 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T05:00:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T05:00:09.225 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:09.225 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-13T05:00:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:09.399 INFO:teuthology.orchestra.run.smithi110.stdout:(1/5): ceph-grafana-dashboards-18.2.4-820.g48a7 154 kB/s | 24 kB 00:00 2024-09-13T05:00:09.425 INFO:teuthology.orchestra.run.smithi110.stdout:(2/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 89 kB/s | 16 kB 00:00 2024-09-13T05:00:09.466 INFO:teuthology.orchestra.run.smithi110.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 458 kB/s | 31 kB 00:00 2024-09-13T05:00:09.508 INFO:teuthology.orchestra.run.smithi110.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-13T05:00:09.683 INFO:teuthology.orchestra.run.smithi110.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 8.0 MB/s | 3.5 MB 00:00 2024-09-13T05:00:09.687 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:09.687 INFO:teuthology.orchestra.run.smithi110.stdout:Total 7.6 MB/s | 3.8 MB 00:00 2024-09-13T05:00:09.687 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:09.694 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:09.694 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:09.779 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:09.779 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:10.027 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:10.123 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T05:00:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.e 1/1 2024-09-13T05:00:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:10.147 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:10.147 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-immutable-object-cache-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:10.148 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:10.148 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:10.163 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T05:00:10.192 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T05:00:10.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr 2024-09-13T05:00:10.958 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:55 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:11.052 INFO:teuthology.orchestra.run.smithi053.stdout:Package ceph-mgr-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:11.096 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:11.097 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:11.097 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:11.167 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-dashboard 2024-09-13T05:00:11.235 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T05:00:11.279 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T05:00:11.753 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:00:56 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:11.892 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 3.5 M 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 24 k 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 16 k 2024-09-13T05:00:11.893 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout:Install 5 Packages 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:11.894 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.8 M 2024-09-13T05:00:11.895 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 87 M 2024-09-13T05:00:11.895 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:12.208 INFO:teuthology.orchestra.run.smithi053.stdout:(1/5): ceph-prometheus-alerts-18.2.4-820.g48a7d 105 kB/s | 16 kB 00:00 2024-09-13T05:00:12.234 INFO:teuthology.orchestra.run.smithi053.stdout:(2/5): ceph-grafana-dashboards-18.2.4-820.g48a7 134 kB/s | 24 kB 00:00 2024-09-13T05:00:12.276 INFO:teuthology.orchestra.run.smithi053.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-13T05:00:12.309 INFO:teuthology.orchestra.run.smithi053.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-13T05:00:12.346 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T05:00:12.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T05:00:12.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T05:00:12.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T05:00:12.346 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T05:00:12.434 INFO:teuthology.orchestra.run.smithi053.stdout:(5/5): ceph-mgr-dashboard-18.2.4-820.g48a7de7e. 9.2 MB/s | 3.5 MB 00:00 2024-09-13T05:00:12.438 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:12.438 INFO:teuthology.orchestra.run.smithi053.stdout:Total 7.0 MB/s | 3.8 MB 00:00 2024-09-13T05:00:12.438 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:12.446 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:12.446 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:12.535 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T05:00:12.597 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T05:00:12.598 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:12.599 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:12.851 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T05:00:12.901 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-13T05:00:12.932 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-13T05:00:12.960 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T05:00:13.410 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:00 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:13.543 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T05:00:13.545 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-09-13T05:00:13.546 INFO:teuthology.orchestra.run.smithi110.stdout:Install 12 Packages 2024-09-13T05:00:13.547 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:13.547 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 38 M 2024-09-13T05:00:13.547 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 212 M 2024-09-13T05:00:13.547 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:14.079 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 4/5 2024-09-13T05:00:14.122 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T05:00:14.166 INFO:teuthology.orchestra.run.smithi110.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.4-82 15 MB/s | 7.4 MB 00:00 2024-09-13T05:00:14.291 INFO:teuthology.orchestra.run.smithi110.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 262 kB/s | 33 kB 00:00 2024-09-13T05:00:14.350 INFO:teuthology.orchestra.run.smithi110.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 285 kB/s | 191 kB 00:00 2024-09-13T05:00:14.450 INFO:teuthology.orchestra.run.smithi110.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 801 kB 00:00 2024-09-13T05:00:14.476 INFO:teuthology.orchestra.run.smithi110.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 145 kB/s | 18 kB 00:00 2024-09-13T05:00:14.534 INFO:teuthology.orchestra.run.smithi110.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 461 kB/s | 38 kB 00:00 2024-09-13T05:00:14.643 INFO:teuthology.orchestra.run.smithi110.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-13T05:00:15.119 INFO:teuthology.orchestra.run.smithi110.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.6 MB/s | 3.0 MB 00:00 2024-09-13T05:00:15.242 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 5/5 2024-09-13T05:00:15.242 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.n 1/5 2024-09-13T05:00:15.242 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2/5 2024-09-13T05:00:15.242 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.no 3/5 2024-09-13T05:00:15.243 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-13T05:00:15.260 INFO:teuthology.orchestra.run.smithi110.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.7 MB/s | 250 kB 00:00 2024-09-13T05:00:15.520 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-13T05:00:15.520 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:15.521 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:15.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-grafana-dashboards-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:15.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-dashboard-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:15.521 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-prometheus-alerts-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:15.521 INFO:teuthology.orchestra.run.smithi053.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-13T05:00:15.521 INFO:teuthology.orchestra.run.smithi053.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-13T05:00:15.522 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:15.522 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:15.729 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-13T05:00:16.294 INFO:teuthology.orchestra.run.smithi110.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.7 MB/s | 4.9 MB 00:01 2024-09-13T05:00:16.319 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:01 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:16.461 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 7.4 M 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-13T05:00:16.462 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout:====================================================================================== 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout:Install 12 Packages 2024-09-13T05:00:16.463 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:16.464 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 38 M 2024-09-13T05:00:16.464 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 212 M 2024-09-13T05:00:16.464 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:16.711 INFO:teuthology.orchestra.run.smithi110.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.5 MB/s | 5.1 MB 00:02 2024-09-13T05:00:16.892 INFO:teuthology.orchestra.run.smithi053.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-13T05:00:16.950 INFO:teuthology.orchestra.run.smithi053.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-13T05:00:17.034 INFO:teuthology.orchestra.run.smithi053.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-09-13T05:00:17.084 INFO:teuthology.orchestra.run.smithi053.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 363 kB/s | 18 kB 00:00 2024-09-13T05:00:17.136 INFO:teuthology.orchestra.run.smithi053.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-13T05:00:17.293 INFO:teuthology.orchestra.run.smithi053.stdout:(6/12): ceph-mgr-diskprediction-local-18.2.4-82 14 MB/s | 7.4 MB 00:00 2024-09-13T05:00:17.518 INFO:teuthology.orchestra.run.smithi053.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-09-13T05:00:18.077 INFO:teuthology.orchestra.run.smithi053.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-13T05:00:18.227 INFO:teuthology.orchestra.run.smithi053.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.6 MB/s | 250 kB 00:00 2024-09-13T05:00:19.103 INFO:teuthology.orchestra.run.smithi053.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.5 MB/s | 4.9 MB 00:01 2024-09-13T05:00:19.528 INFO:teuthology.orchestra.run.smithi053.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.5 MB/s | 5.1 MB 00:02 2024-09-13T05:00:23.094 INFO:teuthology.orchestra.run.smithi110.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-09-13T05:00:23.095 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:23.095 INFO:teuthology.orchestra.run.smithi110.stdout:Total 4.0 MB/s | 38 MB 00:09 2024-09-13T05:00:23.451 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:23.477 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:23.477 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:24.011 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:24.011 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:24.779 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:24.871 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T05:00:24.917 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T05:00:25.016 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T05:00:25.060 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T05:00:25.244 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T05:00:25.275 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T05:00:25.353 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T05:00:25.997 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T05:00:26.094 INFO:teuthology.orchestra.run.smithi053.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-09-13T05:00:26.095 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:26.095 INFO:teuthology.orchestra.run.smithi053.stdout:Total 4.0 MB/s | 38 MB 00:09 2024-09-13T05:00:26.113 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T05:00:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:26.491 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:27.030 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:27.030 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T05:00:27.812 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:27.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-13T05:00:27.938 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-13T05:00:28.039 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-13T05:00:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-13T05:00:28.279 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-13T05:00:28.342 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-13T05:00:28.383 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T05:00:28.426 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T05:00:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-13T05:00:29.128 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-13T05:00:29.242 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-13T05:00:30.575 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T05:00:31.042 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T05:00:31.043 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T05:00:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:31.455 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:31.670 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-13T05:00:31.699 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T05:00:31.872 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-rook 2024-09-13T05:00:32.439 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:19 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:32.571 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:32.572 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:32.572 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:32.572 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:32.572 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:32.572 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T05:00:32.573 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout:Install 13 Packages 2024-09-13T05:00:32.574 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:32.575 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 2.2 M 2024-09-13T05:00:32.575 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 26 M 2024-09-13T05:00:32.575 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:33.144 INFO:teuthology.orchestra.run.smithi110.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 168 kB/s | 19 kB 00:00 2024-09-13T05:00:33.170 INFO:teuthology.orchestra.run.smithi110.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 184 kB/s | 26 kB 00:00 2024-09-13T05:00:33.195 INFO:teuthology.orchestra.run.smithi110.stdout:(3/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 299 kB/s | 49 kB 00:00 2024-09-13T05:00:33.287 INFO:teuthology.orchestra.run.smithi110.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-13T05:00:33.321 INFO:teuthology.orchestra.run.smithi110.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.2 MB/s | 222 kB 00:00 2024-09-13T05:00:33.346 INFO:teuthology.orchestra.run.smithi110.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-09-13T05:00:33.388 INFO:teuthology.orchestra.run.smithi110.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-13T05:00:33.405 INFO:teuthology.orchestra.run.smithi110.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 240 kB/s | 14 kB 00:00 2024-09-13T05:00:33.438 INFO:teuthology.orchestra.run.smithi110.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-13T05:00:33.472 INFO:teuthology.orchestra.run.smithi110.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-13T05:00:33.489 INFO:teuthology.orchestra.run.smithi110.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-13T05:00:33.514 INFO:teuthology.orchestra.run.smithi110.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.2 MB/s | 90 kB 00:00 2024-09-13T05:00:33.548 INFO:teuthology.orchestra.run.smithi110.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-13T05:00:33.551 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:33.551 INFO:teuthology.orchestra.run.smithi110.stdout:Total 2.3 MB/s | 2.2 MB 00:00 2024-09-13T05:00:33.611 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:33.625 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:33.626 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:33.712 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:33.712 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:34.035 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 12/12 2024-09-13T05:00:34.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de 1/12 2024-09-13T05:00:34.035 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-13T05:00:34.036 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-13T05:00:34.233 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:34.402 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T05:00:34.457 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T05:00:34.513 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T05:00:34.515 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-13T05:00:34.515 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-diskprediction-local-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-13T05:00:34.516 INFO:teuthology.orchestra.run.smithi053.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-13T05:00:34.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-13T05:00:34.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-13T05:00:34.517 INFO:teuthology.orchestra.run.smithi053.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-13T05:00:34.517 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:34.517 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:34.559 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T05:00:34.600 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T05:00:34.701 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T05:00:34.812 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T05:00:34.860 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T05:00:34.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-rook 2024-09-13T05:00:35.508 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:20 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:35.579 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T05:00:35.642 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T05:00:35.647 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 49 k 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-13T05:00:35.649 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:35.650 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:35.651 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:35.651 INFO:teuthology.orchestra.run.smithi053.stdout:Install 13 Packages 2024-09-13T05:00:35.651 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:35.651 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 2.2 M 2024-09-13T05:00:35.651 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 26 M 2024-09-13T05:00:35.651 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:35.675 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T05:00:35.723 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T05:00:35.754 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T05:00:35.815 INFO:teuthology.orchestra.run.smithi053.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 226 kB/s | 19 kB 00:00 2024-09-13T05:00:35.840 INFO:teuthology.orchestra.run.smithi053.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 232 kB/s | 26 kB 00:00 2024-09-13T05:00:35.899 INFO:teuthology.orchestra.run.smithi053.stdout:(3/13): ceph-mgr-rook-18.2.4-820.g48a7de7e.el9. 292 kB/s | 49 kB 00:00 2024-09-13T05:00:35.924 INFO:teuthology.orchestra.run.smithi053.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-13T05:00:35.949 INFO:teuthology.orchestra.run.smithi053.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-13T05:00:35.975 INFO:teuthology.orchestra.run.smithi053.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-13T05:00:36.017 INFO:teuthology.orchestra.run.smithi053.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 482 kB/s | 32 kB 00:00 2024-09-13T05:00:36.042 INFO:teuthology.orchestra.run.smithi053.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 211 kB/s | 14 kB 00:00 2024-09-13T05:00:36.067 INFO:teuthology.orchestra.run.smithi053.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-13T05:00:36.118 INFO:teuthology.orchestra.run.smithi053.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.2 MB/s | 220 kB 00:00 2024-09-13T05:00:36.151 INFO:teuthology.orchestra.run.smithi053.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-13T05:00:36.168 INFO:teuthology.orchestra.run.smithi053.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 587 kB/s | 59 kB 00:00 2024-09-13T05:00:36.202 INFO:teuthology.orchestra.run.smithi053.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-13T05:00:36.204 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:36.204 INFO:teuthology.orchestra.run.smithi053.stdout:Total 4.0 MB/s | 2.2 MB 00:00 2024-09-13T05:00:36.269 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:36.285 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:36.286 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:36.383 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:36.385 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T05:00:36.498 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T05:00:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T05:00:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T05:00:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T05:00:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T05:00:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T05:00:36.834 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T05:00:36.835 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T05:00:36.835 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T05:00:36.835 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T05:00:36.835 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T05:00:36.835 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:36.835 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:36.909 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-13T05:00:37.113 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T05:00:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-13T05:00:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-13T05:00:37.263 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-13T05:00:37.305 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-13T05:00:37.412 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-13T05:00:37.530 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-13T05:00:37.590 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-13T05:00:37.678 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:24 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:37.810 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T05:00:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout:Install 6 Packages 2024-09-13T05:00:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:37.813 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 7.0 M 2024-09-13T05:00:37.813 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 33 M 2024-09-13T05:00:37.813 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:38.061 INFO:teuthology.orchestra.run.smithi110.stdout:(1/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 706 kB/s | 139 kB 00:00 2024-09-13T05:00:38.144 INFO:teuthology.orchestra.run.smithi110.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-13T05:00:38.170 INFO:teuthology.orchestra.run.smithi110.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-13T05:00:38.195 INFO:teuthology.orchestra.run.smithi110.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-13T05:00:38.355 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-13T05:00:38.411 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-13T05:00:38.445 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-13T05:00:38.494 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-13T05:00:38.504 INFO:teuthology.orchestra.run.smithi110.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 388 kB/s | 248 kB 00:00 2024-09-13T05:00:38.522 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T05:00:39.237 INFO:teuthology.orchestra.run.smithi110.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.3 MB/s | 6.0 MB 00:01 2024-09-13T05:00:39.238 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:39.238 INFO:teuthology.orchestra.run.smithi110.stdout:Total 4.9 MB/s | 7.0 MB 00:01 2024-09-13T05:00:39.308 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 13/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 1/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-13T05:00:39.313 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-13T05:00:39.314 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-13T05:00:39.319 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:39.319 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:39.421 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:39.423 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-rook-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-13T05:00:39.676 INFO:teuthology.orchestra.run.smithi053.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:39.677 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:39.723 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:39.896 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T05:00:39.942 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-mgr-cephadm 2024-09-13T05:00:39.948 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T05:00:40.479 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T05:00:40.536 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:25 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:40.558 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T05:00:40.620 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T05:00:40.643 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T05:00:40.675 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:40.676 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:40.676 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:40.676 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 139 k 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout:Install 6 Packages 2024-09-13T05:00:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:40.678 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 7.0 M 2024-09-13T05:00:40.678 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 33 M 2024-09-13T05:00:40.678 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:41.062 INFO:teuthology.orchestra.run.smithi053.stdout:(1/6): ceph-mgr-cephadm-18.2.4-820.g48a7de7e.el 612 kB/s | 139 kB 00:00 2024-09-13T05:00:41.088 INFO:teuthology.orchestra.run.smithi053.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 984 kB/s | 248 kB 00:00 2024-09-13T05:00:41.146 INFO:teuthology.orchestra.run.smithi053.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.4 MB/s | 548 kB 00:00 2024-09-13T05:00:41.171 INFO:teuthology.orchestra.run.smithi053.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 691 kB/s | 58 kB 00:00 2024-09-13T05:00:41.197 INFO:teuthology.orchestra.run.smithi053.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-13T05:00:41.372 INFO:teuthology.orchestra.run.smithi053.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-09-13T05:00:41.373 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:41.374 INFO:teuthology.orchestra.run.smithi053.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-13T05:00:41.390 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T05:00:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T05:00:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T05:00:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T05:00:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T05:00:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T05:00:41.447 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:41.458 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:41.459 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:41.564 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:41.565 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:41.661 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T05:00:41.662 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T05:00:41.663 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:41.663 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:41.883 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:41.927 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-fuse 2024-09-13T05:00:42.054 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-13T05:00:42.116 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-13T05:00:42.491 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:29 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:42.623 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:42.625 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:42.625 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:42.625 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout:Install 2 Packages 2024-09-13T05:00:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:42.627 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 932 k 2024-09-13T05:00:42.627 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 2.7 M 2024-09-13T05:00:42.627 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:42.680 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-13T05:00:42.771 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-13T05:00:42.823 INFO:teuthology.orchestra.run.smithi110.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 863 kB/s | 80 kB 00:00 2024-09-13T05:00:42.841 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-13T05:00:42.877 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T05:00:43.007 INFO:teuthology.orchestra.run.smithi110.stdout:(2/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 3.0 MB/s | 852 kB 00:00 2024-09-13T05:00:43.007 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:43.007 INFO:teuthology.orchestra.run.smithi110.stdout:Total 2.4 MB/s | 932 kB 00:00 2024-09-13T05:00:43.019 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:43.040 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:43.040 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:43.120 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:43.120 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:43.314 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:43.455 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T05:00:43.530 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T05:00:43.634 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 6/6 2024-09-13T05:00:43.634 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 1/6 2024-09-13T05:00:43.634 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-13T05:00:43.634 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-13T05:00:43.634 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-13T05:00:43.634 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-mgr-cephadm-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-13T05:00:43.917 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:43.918 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:44.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-fuse 2024-09-13T05:00:44.261 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T05:00:44.261 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T05:00:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T05:00:44.542 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:44.542 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T05:00:44.543 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:44.543 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:44.767 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:29 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:44.846 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-volume 2024-09-13T05:00:44.906 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 852 k 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout:Install 2 Packages 2024-09-13T05:00:44.907 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:44.908 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 932 k 2024-09-13T05:00:44.908 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 2.7 M 2024-09-13T05:00:44.908 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:45.281 INFO:teuthology.orchestra.run.smithi053.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 688 kB/s | 80 kB 00:00 2024-09-13T05:00:45.423 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:32 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:45.440 INFO:teuthology.orchestra.run.smithi053.stdout:(2/2): ceph-fuse-18.2.4-820.g48a7de7e.el9.x86_6 3.0 MB/s | 852 kB 00:00 2024-09-13T05:00:45.440 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:45.441 INFO:teuthology.orchestra.run.smithi053.stdout:Total 1.7 MB/s | 932 kB 00:00 2024-09-13T05:00:45.453 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:45.476 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:45.476 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:45.560 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:45.560 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout:Install 2 Packages 2024-09-13T05:00:45.561 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:45.562 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 342 k 2024-09-13T05:00:45.562 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 1.5 M 2024-09-13T05:00:45.562 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:45.563 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:45.563 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:45.757 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:45.893 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-13T05:00:45.975 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T05:00:46.054 INFO:teuthology.orchestra.run.smithi110.stdout:(1/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-13T05:00:46.471 INFO:teuthology.orchestra.run.smithi110.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 118 kB/s | 77 kB 00:00 2024-09-13T05:00:46.472 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:46.472 INFO:teuthology.orchestra.run.smithi110.stdout:Total 375 kB/s | 342 kB 00:00 2024-09-13T05:00:46.485 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:46.495 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:46.496 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:46.531 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:46.531 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:46.668 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2/2 2024-09-13T05:00:46.668 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/2 2024-09-13T05:00:46.680 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:46.872 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T05:00:46.907 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T05:00:46.922 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T05:00:46.922 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:46.922 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T05:00:46.923 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-13T05:00:46.964 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:46.965 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-13T05:00:46.965 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:46.965 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:47.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install ceph-volume 2024-09-13T05:00:47.654 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T05:00:47.859 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:32 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:47.970 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T05:00:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:47.971 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T05:00:47.971 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:47.971 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repository Size 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume noarch 2:18.2.4-820.g48a7de7e.el9 ceph-noarch 264 k 2024-09-13T05:00:47.998 INFO:teuthology.orchestra.run.smithi053.stdout:Installing dependencies: 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout:Install 2 Packages 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:47.999 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 342 k 2024-09-13T05:00:48.000 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 1.5 M 2024-09-13T05:00:48.000 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:48.261 INFO:teuthology.orchestra.run.smithi053.stdout:(1/2): ceph-volume-18.2.4-820.g48a7de7e.el9.noa 1.1 MB/s | 264 kB 00:00 2024-09-13T05:00:48.330 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados-devel 2024-09-13T05:00:48.453 INFO:teuthology.orchestra.run.smithi053.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 179 kB/s | 77 kB 00:00 2024-09-13T05:00:48.454 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:48.454 INFO:teuthology.orchestra.run.smithi053.stdout:Total 751 kB/s | 342 kB 00:00 2024-09-13T05:00:48.465 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:48.476 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:48.476 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:48.510 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:48.510 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:48.660 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:48.856 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-13T05:00:48.889 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T05:00:48.905 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2/2 2024-09-13T05:00:48.905 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:00:48.906 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-13T05:00:48.906 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:48.907 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:49.042 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:49.043 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:49.043 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-13T05:00:49.043 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 127 k 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 456 k 2024-09-13T05:00:49.044 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:49.260 INFO:teuthology.orchestra.run.smithi110.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 586 kB/s | 127 kB 00:00 2024-09-13T05:00:49.261 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:49.261 INFO:teuthology.orchestra.run.smithi110.stdout:Total 582 kB/s | 127 kB 00:00 2024-09-13T05:00:49.261 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:49.267 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:49.267 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:49.331 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:49.331 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:49.498 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:49.597 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 1/2 2024-09-13T05:00:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:49.933 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-13T05:00:49.933 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:49.934 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:49.934 INFO:teuthology.orchestra.run.smithi053.stdout: ceph-volume-2:18.2.4-820.g48a7de7e.el9.noarch 2024-09-13T05:00:49.934 INFO:teuthology.orchestra.run.smithi053.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-13T05:00:49.934 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:49.934 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:50.075 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:50.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados-devel 2024-09-13T05:00:50.326 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:50.326 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:50.326 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:50.326 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:50.327 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:50.327 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:50.601 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs2 2024-09-13T05:00:50.836 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:35 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:50.975 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 127 k 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:50.976 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:00:50.977 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 127 k 2024-09-13T05:00:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 456 k 2024-09-13T05:00:50.977 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:51.163 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:51.183 INFO:teuthology.orchestra.run.smithi053.stdout:librados-devel-18.2.4-820.g48a7de7e.el9.x86_64. 616 kB/s | 127 kB 00:00 2024-09-13T05:00:51.183 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:51.183 INFO:teuthology.orchestra.run.smithi053.stdout:Total 611 kB/s | 127 kB 00:00 2024-09-13T05:00:51.184 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:51.189 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:51.254 INFO:teuthology.orchestra.run.smithi110.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:51.257 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:51.257 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:51.299 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:51.300 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:51.300 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:51.392 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs-devel 2024-09-13T05:00:51.413 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:51.533 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:51.946 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:51.958 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:52.079 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:52.080 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:52.080 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:00:52.080 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:52.080 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 31 k 2024-09-13T05:00:52.080 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 139 k 2024-09-13T05:00:52.080 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:52.216 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:52.216 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:52.217 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:52.217 INFO:teuthology.orchestra.run.smithi053.stdout: librados-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:52.217 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:52.217 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:52.231 INFO:teuthology.orchestra.run.smithi110.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 207 kB/s | 31 kB 00:00 2024-09-13T05:00:52.231 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:52.232 INFO:teuthology.orchestra.run.smithi110.stdout:Total 204 kB/s | 31 kB 00:00 2024-09-13T05:00:52.232 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:52.253 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:52.253 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:52.345 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:52.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs2 2024-09-13T05:00:52.469 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:52.834 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:53.038 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:53.060 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:53.060 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:53.060 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:53.060 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:53.060 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:53.060 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:53.133 INFO:teuthology.orchestra.run.smithi053.stdout:Package libcephfs2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:53.178 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:53.179 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:53.179 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:53.263 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install libcephfs-devel 2024-09-13T05:00:53.278 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados2 2024-09-13T05:00:53.842 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:40 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:53.852 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:38 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:53.933 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:53.975 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:53.976 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:53.976 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:53.990 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:53.990 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 31 k 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 31 k 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 139 k 2024-09-13T05:00:53.991 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:00:54.067 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librbd1 2024-09-13T05:00:54.153 INFO:teuthology.orchestra.run.smithi053.stdout:libcephfs-devel-18.2.4-820.g48a7de7e.el9.x86_64 193 kB/s | 31 kB 00:00 2024-09-13T05:00:54.154 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:54.154 INFO:teuthology.orchestra.run.smithi053.stdout:Total 191 kB/s | 31 kB 00:00 2024-09-13T05:00:54.154 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:00:54.157 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:00:54.158 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:00:54.174 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:00:54.175 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:00:54.266 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:00:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:54.629 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:41 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:54.719 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:54.744 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:54.762 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:54.763 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:54.763 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:54.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rados 2024-09-13T05:00:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:54.989 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:00:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: libcephfs-devel-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:00:54.990 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:55.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librados2 2024-09-13T05:00:55.410 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:55.499 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:55.542 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:55.543 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:55.543 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:55.630 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rgw 2024-09-13T05:00:55.761 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:40 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:55.856 INFO:teuthology.orchestra.run.smithi053.stdout:Package librados2-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:55.902 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:55.902 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:55.903 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:55.967 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install librbd1 2024-09-13T05:00:56.191 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:56.281 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:56.323 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:56.324 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:56.324 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:56.416 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-cephfs 2024-09-13T05:00:56.557 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:41 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:56.650 INFO:teuthology.orchestra.run.smithi053.stdout:Package librbd1-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:56.695 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:56.696 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:56.696 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:56.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rados 2024-09-13T05:00:56.976 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:57.065 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:57.107 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:57.108 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:57.108 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:57.196 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rbd 2024-09-13T05:00:57.360 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:42 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:57.457 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rados-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:57.502 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:57.503 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:57.503 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:57.575 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rgw 2024-09-13T05:00:57.757 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:44 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:57.846 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:57.888 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-09-13T05:00:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:57.973 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-fuse 2024-09-13T05:00:58.160 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:58.253 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rgw-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:58.299 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:58.300 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:58.301 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:58.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-cephfs 2024-09-13T05:00:58.534 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:45 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:00:58.665 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:00:58.666 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:58.666 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-13T05:00:58.666 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:58.666 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:00:58.666 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 87 k 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 227 k 2024-09-13T05:00:58.667 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:00:58.876 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 415 kB/s | 87 kB 00:00 2024-09-13T05:00:58.877 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:00:58.877 INFO:teuthology.orchestra.run.smithi110.stdout:Total 412 kB/s | 87 kB 00:00 2024-09-13T05:00:58.877 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:00:58.884 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:00:58.884 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:00:58.949 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:00:58.949 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:00:58.959 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:43 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:59.051 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-cephfs-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:59.091 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:00:59.097 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:59.098 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:59.098 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:59.170 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install python3-rbd 2024-09-13T05:00:59.190 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:59.594 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:59.753 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:44 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:00:59.846 INFO:teuthology.orchestra.run.smithi053.stdout:Package python3-rbd-2:18.2.4-820.g48a7de7e.el9.x86_64 is already installed. 2024-09-13T05:00:59.847 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:00:59.847 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:59.847 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:00:59.847 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:00:59.847 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:00:59.847 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:00:59.892 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:00:59.893 INFO:teuthology.orchestra.run.smithi053.stdout:Nothing to do. 2024-09-13T05:00:59.893 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:00:59.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-fuse 2024-09-13T05:01:00.148 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-mirror 2024-09-13T05:01:00.560 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:45 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:01:00.702 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 87 k 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:01:00.703 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:00.704 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:01:00.704 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:00.704 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 87 k 2024-09-13T05:01:00.704 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 227 k 2024-09-13T05:01:00.704 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:01:00.723 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:47 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:01:00.858 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:01:00.859 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:00.860 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 3.0 M 2024-09-13T05:01:00.860 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 12 M 2024-09-13T05:01:00.860 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:01:00.897 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-fuse-18.2.4-820.g48a7de7e.el9.x86_64.rpm 449 kB/s | 87 kB 00:00 2024-09-13T05:01:00.898 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:01:00.898 INFO:teuthology.orchestra.run.smithi053.stdout:Total 446 kB/s | 87 kB 00:00 2024-09-13T05:01:00.898 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:01:00.906 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:01:00.906 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:01:00.973 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:01:00.973 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:01:01.124 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:01:01.198 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 8.9 MB/s | 3.0 MB 00:00 2024-09-13T05:01:01.199 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:01:01.199 INFO:teuthology.orchestra.run.smithi110.stdout:Total 8.8 MB/s | 3.0 MB 00:00 2024-09-13T05:01:01.200 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:01:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:01:01.209 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:01:01.239 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:01.287 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:01:01.288 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:01:01.632 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:01.705 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:01:01.774 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:01.801 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:01.801 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:01:01.802 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T05:01:01.802 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T05:01:01.802 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T05:01:01.802 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-fuse-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:01.949 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:01:02.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-mirror 2024-09-13T05:01:02.814 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:47 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:01:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:02.891 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:01:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:01:02.891 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:02.891 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:01:02.954 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout: Package Arch Version Repo Size 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 3.0 M 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:02.955 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 3.0 M 2024-09-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 12 M 2024-09-13T05:01:02.956 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:01:03.154 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-nbd 2024-09-13T05:01:03.319 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-mirror-18.2.4-820.g48a7de7e.el9.x86_64.rpm 8.2 MB/s | 3.0 MB 00:00 2024-09-13T05:01:03.320 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:01:03.320 INFO:teuthology.orchestra.run.smithi053.stdout:Total 8.2 MB/s | 3.0 MB 00:00 2024-09-13T05:01:03.320 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:01:03.330 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:01:03.330 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:01:03.415 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:01:03.415 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:01:03.734 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:50 ago on Fri 13 Sep 2024 04:59:13 AM UTC. 2024-09-13T05:01:03.853 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:03.869 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-09-13T05:01:03.870 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-09-13T05:01:03.870 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-09-13T05:01:03.870 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:03.870 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 172 k 2024-09-13T05:01:03.870 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 494 k 2024-09-13T05:01:03.870 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-09-13T05:01:03.928 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:03.955 INFO:teuthology.orchestra.run.smithi053.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-13T05:01:03.955 INFO:teuthology.orchestra.run.smithi053.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-13T05:01:03.955 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T05:01:03.955 INFO:teuthology.orchestra.run.smithi053.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-13T05:01:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:04.078 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 829 kB/s | 172 kB 00:00 2024-09-13T05:01:04.078 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:01:04.078 INFO:teuthology.orchestra.run.smithi110.stdout:Total 824 kB/s | 172 kB 00:00 2024-09-13T05:01:04.079 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-09-13T05:01:04.086 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-09-13T05:01:04.086 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-09-13T05:01:04.150 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-09-13T05:01:04.150 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-09-13T05:01:04.307 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-09-13T05:01:04.416 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:04.848 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:05.066 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:01:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-mirror-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:01:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:05.067 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:01:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:05.122 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-09-13T05:01:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:01:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:05.122 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-09-13T05:01:05.306 DEBUG:teuthology.orchestra.run.smithi053:> sudo yum -y install rbd-nbd 2024-09-13T05:01:05.374 DEBUG:teuthology.parallel:result is None 2024-09-13T05:01:05.900 INFO:teuthology.orchestra.run.smithi053.stdout:Last metadata expiration check: 0:01:50 ago on Fri 13 Sep 2024 04:59:15 AM UTC. 2024-09-13T05:01:06.039 INFO:teuthology.orchestra.run.smithi053.stdout:Dependencies resolved. 2024-09-13T05:01:06.039 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:06.039 INFO:teuthology.orchestra.run.smithi053.stdout: Package Architecture Version Repository Size 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout:Installing: 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd x86_64 2:18.2.4-820.g48a7de7e.el9 ceph 172 k 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction Summary 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout:================================================================================ 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout:Install 1 Package 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:06.040 INFO:teuthology.orchestra.run.smithi053.stdout:Total download size: 172 k 2024-09-13T05:01:06.041 INFO:teuthology.orchestra.run.smithi053.stdout:Installed size: 494 k 2024-09-13T05:01:06.041 INFO:teuthology.orchestra.run.smithi053.stdout:Downloading Packages: 2024-09-13T05:01:06.414 INFO:teuthology.orchestra.run.smithi053.stdout:rbd-nbd-18.2.4-820.g48a7de7e.el9.x86_64.rpm 460 kB/s | 172 kB 00:00 2024-09-13T05:01:06.415 INFO:teuthology.orchestra.run.smithi053.stdout:-------------------------------------------------------------------------------- 2024-09-13T05:01:06.415 INFO:teuthology.orchestra.run.smithi053.stdout:Total 458 kB/s | 172 kB 00:00 2024-09-13T05:01:06.415 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction check 2024-09-13T05:01:06.424 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction check succeeded. 2024-09-13T05:01:06.424 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction test 2024-09-13T05:01:06.492 INFO:teuthology.orchestra.run.smithi053.stdout:Transaction test succeeded. 2024-09-13T05:01:06.492 INFO:teuthology.orchestra.run.smithi053.stdout:Running transaction 2024-09-13T05:01:06.648 INFO:teuthology.orchestra.run.smithi053.stdout: Preparing : 1/1 2024-09-13T05:01:06.764 INFO:teuthology.orchestra.run.smithi053.stdout: Installing : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:07.180 INFO:teuthology.orchestra.run.smithi053.stdout: Running scriptlet: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:07.473 INFO:teuthology.orchestra.run.smithi053.stdout: Verifying : rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 1/1 2024-09-13T05:01:07.473 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:07.474 INFO:teuthology.orchestra.run.smithi053.stdout:Installed: 2024-09-13T05:01:07.474 INFO:teuthology.orchestra.run.smithi053.stdout: rbd-nbd-2:18.2.4-820.g48a7de7e.el9.x86_64 2024-09-13T05:01:07.474 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:07.474 INFO:teuthology.orchestra.run.smithi053.stdout:Complete! 2024-09-13T05:01:07.726 DEBUG:teuthology.parallel:result is None 2024-09-13T05:01:07.727 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T05:01:07.892 DEBUG:teuthology.orchestra.run.smithi053:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T05:01:07.915 INFO:teuthology.orchestra.run.smithi053.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T05:01:07.915 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T05:01:07.915 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T05:01:07.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=48a7de7e6e1701a88fd571c7e34908ac4b8ed412 2024-09-13T05:01:08.070 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-13T05:01:08.096 INFO:teuthology.orchestra.run.smithi110.stdout:18.2.4-820.g48a7de7e.el9 2024-09-13T05:01:08.096 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-820.g48a7de7e.el9 2024-09-13T05:01:08.096 INFO:teuthology.task.install:The correct ceph version 18.2.4-820.g48a7de7e is installed. 2024-09-13T05:01:08.098 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-13T05:01:08.099 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:08.099 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T05:01:08.127 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:08.128 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-13T05:01:08.165 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-13T05:01:08.165 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:08.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T05:01:08.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T05:01:08.262 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:08.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-09-13T05:01:08.293 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-13T05:01:08.363 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-13T05:01:08.363 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:08.363 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T05:01:08.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T05:01:08.457 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:08.457 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-13T05:01:08.486 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-13T05:01:08.555 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-13T05:01:08.556 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:08.556 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T05:01:08.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T05:01:08.651 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:08.651 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-09-13T05:01:08.679 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-13T05:01:08.748 INFO:teuthology.run_tasks:Running task ceph... 2024-09-13T05:01:08.832 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-13T05:01:08.832 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 777 /var/log/ceph 2024-09-13T05:01:08.835 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /var/log/ceph 2024-09-13T05:01:08.863 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-13T05:01:08.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T05:01:08.904 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-13T05:01:08.931 INFO:tasks.ceph:Creating extra log directories... 2024-09-13T05:01:08.932 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T05:01:08.972 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-13T05:01:09.010 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-13T05:01:09.010 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-13T05:01:09.010 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7900839', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 248, 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412', 'sleep_before_teardown': 0, 'subset': '116/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7a853b1706d076d58e0da7a5d3b9b9eb36276880/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880', 'targets': {'smithi053.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKcG2CsDwrjUecfaQHjCXiPEjQvoYensJcPHU7fDgWYLerQYHD5dkg9yc9ci2NuW9n+85EZqXJnK8h75gohcN/U=', 'smithi110.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKl6daA7j4Mi0fXzl8iNkxEOCNZ9u5k6SXapUdm7NyGwKKVaHYP0L58v3v5cSyUhV7uwHko1oQQm1C1Cw4WF5jQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_22:16:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-13T05:01:09.010 DEBUG:teuthology.orchestra.run.smithi053:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T05:01:09.041 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-13T05:01:09.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T05:01:09.099 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m0777 -- /var/run/ceph 2024-09-13T05:01:09.126 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:09.126 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T05:01:09.182 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T05:01:09.182 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_1 2024-09-13T05:01:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T05:01:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:09.238 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-13T05:01:09.239 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:09.239 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:09.239 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-13 05:01:04.216863336 +0000 2024-09-13T05:01:09.239 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-13 04:58:35.929076311 +0000 2024-09-13T05:01:09.239 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-13 04:58:35.929076311 +0000 2024-09-13T05:01:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-13 04:58:35.929076311 +0000 2024-09-13T05:01:09.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T05:01:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-13T05:01:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-13T05:01:09.304 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000105129 s, 4.9 MB/s 2024-09-13T05:01:09.305 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T05:01:09.363 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_2 2024-09-13T05:01:09.419 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-13 05:01:04.217863316 +0000 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-13 04:58:36.259069774 +0000 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-13 04:58:36.259069774 +0000 2024-09-13T05:01:09.420 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-13 04:58:36.259069774 +0000 2024-09-13T05:01:09.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T05:01:09.484 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-13T05:01:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-13T05:01:09.485 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000105019 s, 4.9 MB/s 2024-09-13T05:01:09.486 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T05:01:09.543 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_3 2024-09-13T05:01:09.601 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T05:01:09.601 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:09.601 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-13T05:01:09.602 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:09.602 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:09.602 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-13 05:01:04.217863316 +0000 2024-09-13T05:01:09.602 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-13 04:58:36.580063415 +0000 2024-09-13T05:01:09.602 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-13 04:58:36.580063415 +0000 2024-09-13T05:01:09.602 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-13 04:58:36.580063415 +0000 2024-09-13T05:01:09.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T05:01:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-13T05:01:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-13T05:01:09.666 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000103984 s, 4.9 MB/s 2024-09-13T05:01:09.667 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T05:01:09.724 DEBUG:teuthology.orchestra.run.smithi053:> stat /dev/vg_nvme/lv_4 2024-09-13T05:01:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout:Access: 2024-09-13 05:01:04.218863296 +0000 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout:Modify: 2024-09-13 04:58:36.898057116 +0000 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout:Change: 2024-09-13 04:58:36.898057116 +0000 2024-09-13T05:01:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: Birth: 2024-09-13 04:58:36.898057116 +0000 2024-09-13T05:01:09.782 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T05:01:09.845 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records in 2024-09-13T05:01:09.845 INFO:teuthology.orchestra.run.smithi053.stderr:1+0 records out 2024-09-13T05:01:09.845 INFO:teuthology.orchestra.run.smithi053.stderr:512 bytes copied, 0.000100948 s, 5.1 MB/s 2024-09-13T05:01:09.846 DEBUG:teuthology.orchestra.run.smithi053:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T05:01:09.903 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:09.903 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:09.903 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-09-13T05:01:09.923 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-13T05:01:09.923 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-09-13T05:01:09.981 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-13T05:01:09.981 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-13 05:01:02.062563395 +0000 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-13 04:58:30.821332420 +0000 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-13 04:58:30.821332420 +0000 2024-09-13T05:01:09.982 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-13 04:58:30.821332420 +0000 2024-09-13T05:01:09.982 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-13T05:01:10.047 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-13T05:01:10.047 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-13T05:01:10.047 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.0001137 s, 4.5 MB/s 2024-09-13T05:01:10.048 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-13T05:01:10.106 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-13 05:01:02.063563379 +0000 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-13 04:58:31.120327735 +0000 2024-09-13T05:01:10.163 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-13 04:58:31.120327735 +0000 2024-09-13T05:01:10.164 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-13 04:58:31.120327735 +0000 2024-09-13T05:01:10.164 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-13T05:01:10.227 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-13T05:01:10.227 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-13T05:01:10.227 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.00010948 s, 4.7 MB/s 2024-09-13T05:01:10.228 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-13T05:01:10.285 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-09-13T05:01:10.341 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-13T05:01:10.341 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:10.341 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-13T05:01:10.341 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:10.342 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:10.342 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-13 05:01:02.063563379 +0000 2024-09-13T05:01:10.342 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-13 04:58:31.443322675 +0000 2024-09-13T05:01:10.342 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-13 04:58:31.443322675 +0000 2024-09-13T05:01:10.342 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-13 04:58:31.443322675 +0000 2024-09-13T05:01:10.342 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-13T05:01:10.405 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-13T05:01:10.405 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-13T05:01:10.405 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000107699 s, 4.8 MB/s 2024-09-13T05:01:10.407 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-13T05:01:10.463 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-09-13T05:01:10.518 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-13T05:01:10.518 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-13T05:01:10.518 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-13T05:01:10.518 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-13T05:01:10.518 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-09-13T05:01:10.518 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-09-13 05:01:02.063563379 +0000 2024-09-13T05:01:10.519 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-09-13 04:58:31.751317851 +0000 2024-09-13T05:01:10.519 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-09-13 04:58:31.751317851 +0000 2024-09-13T05:01:10.519 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-09-13 04:58:31.751317851 +0000 2024-09-13T05:01:10.519 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-13T05:01:10.582 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-09-13T05:01:10.582 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-09-13T05:01:10.583 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000101485 s, 5.0 MB/s 2024-09-13T05:01:10.584 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-13T05:01:10.640 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:10.640 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T05:01:10.640 INFO:tasks.ceph:Generating config... 2024-09-13T05:01:10.641 INFO:tasks.ceph:[client] rbd cache = True 2024-09-13T05:01:10.642 INFO:tasks.ceph:[client] rbd cache max dirty = 0 2024-09-13T05:01:10.642 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-13T05:01:10.642 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-13T05:01:10.642 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-13T05:01:10.642 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-13T05:01:10.642 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-13T05:01:10.642 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-13T05:01:10.642 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-13T05:01:10.642 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-13T05:01:10.642 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-13T05:01:10.642 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-13T05:01:10.642 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-13T05:01:10.642 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-13T05:01:10.642 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-13T05:01:10.643 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-13T05:01:10.644 INFO:tasks.ceph:Setting up mon.a... 2024-09-13T05:01:10.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-13T05:01:10.696 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.keyring 2024-09-13T05:01:10.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-13T05:01:10.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T05:01:10.774 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.53'), ('mon.c', '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]'), ('mon.b', '172.21.15.110')] 2024-09-13T05:01:10.775 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.110', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache max dirty': 0}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-13T05:01:10.776 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:10.776 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T05:01:10.831 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.53 --addv c '[v2:172.21.15.53:3301,v1:172.21.15.53:6790]' --add b 172.21.15.110 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:10.919 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: generated fsid 01e1476b-1467-4e0e-b01c-0f8cdf8276fe 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:setting min_mon_release = pacific 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 0 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 01e1476b-1467-4e0e-b01c-0f8cdf8276fe 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:last_changed 2024-09-13T05:01:10.919957+0000 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-09-13T05:01:10.919957+0000 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:min_mon_release 16 (pacific) 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:election_strategy: 1 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:0: [v2:172.21.15.53:3300/0,v1:172.21.15.53:6789/0] mon.a 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.b 2024-09-13T05:01:10.920 INFO:teuthology.orchestra.run.smithi053.stdout:2: [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] mon.c 2024-09-13T05:01:10.921 INFO:teuthology.orchestra.run.smithi053.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-13T05:01:10.922 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-13T05:01:10.937 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 01e1476b-1467-4e0e-b01c-0f8cdf8276fe... 2024-09-13T05:01:10.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T05:01:10.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-13T05:01:11.022 INFO:teuthology.orchestra.run.smithi053.stdout:[global] 2024-09-13T05:01:11.022 INFO:teuthology.orchestra.run.smithi053.stdout: chdir = "" 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: auth supported = cephx 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: filestore xattr use omap = true 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: mon clock drift allowed = 1.000 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: osd crush chooseleaf type = 0 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: auth debug = true 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on old message = true 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: ms die on bug = true 2024-09-13T05:01:11.023 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon pg warn max object skew = 0 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default size = 2 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow primary affinity = true 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd allow pg remap = true 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on legacy crush tunables = false 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on crush straw calc version zero = false 2024-09-13T05:01:11.024 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on no sortbitwise = false 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on osd down out interval zero = false 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on too few osds = false 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon_allow_pool_size_one = true 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: osd default data pool replay window = 5 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon allow pool delete = true 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.025 INFO:teuthology.orchestra.run.smithi053.stdout: mon cluster log file level = debug 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: debug asserts on shutdown = true 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: mon health detail to clog = false 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: mon host = "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.110" 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: mon client directed command retry = 5 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: mon warn on pool no app = False 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: ms inject socket failures = 5000 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: fsid = 01e1476b-1467-4e0e-b01c-0f8cdf8276fe 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout:[osd] 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: osd journal size = 100 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub load threshold = 5.0 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: osd scrub max interval = 600 2024-09-13T05:01:11.026 INFO:teuthology.orchestra.run.smithi053.stdout: osd mclock profile = high_recovery_ops 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd recover clone overlap = true 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd recovery max chunk = 1048576 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug shutdown = true 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug op order = true 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug verify stray on activate = true 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd open classes on start = true 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug pg log writeout = true 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: osd deep scrub update digest min age = 30 2024-09-13T05:01:11.027 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: osd map max advance = 10 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: journal zero on create = true 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: filestore ondisk finisher threads = 3 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: filestore apply finisher threads = 3 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: bdev debug aio = true 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: osd debug misdirected ops = true 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore block size = 96636764160 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore compression algorithm = zstd 2024-09-13T05:01:11.028 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore compression mode = aggressive 2024-09-13T05:01:11.029 INFO:teuthology.orchestra.run.smithi053.stdout: bluestore fsck on mount = True 2024-09-13T05:01:11.029 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluefs = 1/20 2024-09-13T05:01:11.029 INFO:teuthology.orchestra.run.smithi053.stdout: debug bluestore = 1/20 2024-09-13T05:01:11.029 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-13T05:01:11.030 INFO:teuthology.orchestra.run.smithi053.stdout: debug osd = 20 2024-09-13T05:01:11.030 INFO:teuthology.orchestra.run.smithi053.stdout: debug rocksdb = 4/10 2024-09-13T05:01:11.030 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T05:01:11.030 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd full ratio = 0.9 2024-09-13T05:01:11.030 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd nearfull ratio = 0.8 2024-09-13T05:01:11.031 INFO:teuthology.orchestra.run.smithi053.stdout: osd blocked scrub grace period = 3600 2024-09-13T05:01:11.031 INFO:teuthology.orchestra.run.smithi053.stdout: osd failsafe full ratio = 0.95 2024-09-13T05:01:11.031 INFO:teuthology.orchestra.run.smithi053.stdout: osd max markdown count = 1000 2024-09-13T05:01:11.031 INFO:teuthology.orchestra.run.smithi053.stdout: osd objectstore = bluestore 2024-09-13T05:01:11.031 INFO:teuthology.orchestra.run.smithi053.stdout: osd shutdown pgref assert = True 2024-09-13T05:01:11.031 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.032 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr] 2024-09-13T05:01:11.032 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-13T05:01:11.032 INFO:teuthology.orchestra.run.smithi053.stdout: debug mgr = 20 2024-09-13T05:01:11.032 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-09-13T05:01:11.032 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-09-13T05:01:11.033 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-09-13T05:01:11.033 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-09-13T05:01:11.033 INFO:teuthology.orchestra.run.smithi053.stdout: mgr/telemetry/nag = false 2024-09-13T05:01:11.033 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.033 INFO:teuthology.orchestra.run.smithi053.stdout:[mon] 2024-09-13T05:01:11.033 INFO:teuthology.orchestra.run.smithi053.stdout: debug ms = 1 2024-09-13T05:01:11.034 INFO:teuthology.orchestra.run.smithi053.stdout: debug mon = 20 2024-09-13T05:01:11.034 INFO:teuthology.orchestra.run.smithi053.stdout: debug paxos = 20 2024-09-13T05:01:11.034 INFO:teuthology.orchestra.run.smithi053.stdout: debug auth = 20 2024-09-13T05:01:11.034 INFO:teuthology.orchestra.run.smithi053.stdout: mon data avail warn = 5 2024-09-13T05:01:11.034 INFO:teuthology.orchestra.run.smithi053.stdout: mon mgr mkfs grace = 240 2024-09-13T05:01:11.034 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min pgs per osd = 4 2024-09-13T05:01:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd reporter subtree level = osd 2024-09-13T05:01:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: mon osd prime pg temp = true 2024-09-13T05:01:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: mon reweight min bytes per osd = 10 2024-09-13T05:01:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.035 INFO:teuthology.orchestra.run.smithi053.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T05:01:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T05:01:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T05:01:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T05:01:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T05:01:11.036 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T05:01:11.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.037 INFO:teuthology.orchestra.run.smithi053.stdout: # 1m isn't quite enough 2024-09-13T05:01:11.037 INFO:teuthology.orchestra.run.smithi053.stdout: mon_down_mkfs_grace = 2m 2024-09-13T05:01:11.037 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.037 INFO:teuthology.orchestra.run.smithi053.stdout: mon_warn_on_filestore_osds = false 2024-09-13T05:01:11.038 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:11.038 INFO:teuthology.orchestra.run.smithi053.stdout:[client] 2024-09-13T05:01:11.038 INFO:teuthology.orchestra.run.smithi053.stdout: rgw cache enabled = true 2024-09-13T05:01:11.038 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable ops log = true 2024-09-13T05:01:11.038 INFO:teuthology.orchestra.run.smithi053.stdout: rgw enable usage log = true 2024-09-13T05:01:11.038 INFO:teuthology.orchestra.run.smithi053.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T05:01:11.039 INFO:teuthology.orchestra.run.smithi053.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T05:01:11.039 INFO:teuthology.orchestra.run.smithi053.stdout: rbd cache = True 2024-09-13T05:01:11.039 INFO:teuthology.orchestra.run.smithi053.stdout: rbd cache max dirty = 0 2024-09-13T05:01:11.039 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.a] 2024-09-13T05:01:11.039 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.c] 2024-09-13T05:01:11.039 INFO:teuthology.orchestra.run.smithi053.stdout:[mon.b] 2024-09-13T05:01:11.042 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-09-13T05:01:11.042 INFO:teuthology.orchestra.run.smithi110.stdout: chdir = "" 2024-09-13T05:01:11.043 INFO:teuthology.orchestra.run.smithi110.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-13T05:01:11.043 INFO:teuthology.orchestra.run.smithi110.stdout: auth supported = cephx 2024-09-13T05:01:11.043 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.043 INFO:teuthology.orchestra.run.smithi110.stdout: filestore xattr use omap = true 2024-09-13T05:01:11.043 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.044 INFO:teuthology.orchestra.run.smithi110.stdout: mon clock drift allowed = 1.000 2024-09-13T05:01:11.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.044 INFO:teuthology.orchestra.run.smithi110.stdout: osd crush chooseleaf type = 0 2024-09-13T05:01:11.044 INFO:teuthology.orchestra.run.smithi110.stdout: auth debug = true 2024-09-13T05:01:11.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.045 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on old message = true 2024-09-13T05:01:11.045 INFO:teuthology.orchestra.run.smithi110.stdout: ms die on bug = true 2024-09-13T05:01:11.045 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.045 INFO:teuthology.orchestra.run.smithi110.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-13T05:01:11.045 INFO:teuthology.orchestra.run.smithi110.stdout: mon pg warn max object skew = 0 2024-09-13T05:01:11.046 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.046 INFO:teuthology.orchestra.run.smithi110.stdout: # disable pg_autoscaler by default for new pools 2024-09-13T05:01:11.046 INFO:teuthology.orchestra.run.smithi110.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-13T05:01:11.046 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.046 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default size = 2 2024-09-13T05:01:11.046 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.047 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow primary affinity = true 2024-09-13T05:01:11.047 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd allow pg remap = true 2024-09-13T05:01:11.047 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on legacy crush tunables = false 2024-09-13T05:01:11.047 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on crush straw calc version zero = false 2024-09-13T05:01:11.047 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on no sortbitwise = false 2024-09-13T05:01:11.048 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on osd down out interval zero = false 2024-09-13T05:01:11.048 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on too few osds = false 2024-09-13T05:01:11.048 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-13T05:01:11.048 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-13T05:01:11.048 INFO:teuthology.orchestra.run.smithi110.stdout: mon_allow_pool_size_one = true 2024-09-13T05:01:11.048 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.049 INFO:teuthology.orchestra.run.smithi110.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-13T05:01:11.049 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.049 INFO:teuthology.orchestra.run.smithi110.stdout: osd default data pool replay window = 5 2024-09-13T05:01:11.049 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.049 INFO:teuthology.orchestra.run.smithi110.stdout: mon allow pool delete = true 2024-09-13T05:01:11.050 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.050 INFO:teuthology.orchestra.run.smithi110.stdout: mon cluster log file level = debug 2024-09-13T05:01:11.050 INFO:teuthology.orchestra.run.smithi110.stdout: debug asserts on shutdown = true 2024-09-13T05:01:11.050 INFO:teuthology.orchestra.run.smithi110.stdout: mon health detail to clog = false 2024-09-13T05:01:11.050 INFO:teuthology.orchestra.run.smithi110.stdout: mon host = "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.110" 2024-09-13T05:01:11.050 INFO:teuthology.orchestra.run.smithi110.stdout: mon client directed command retry = 5 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: mon warn on pool no app = False 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: ms inject socket failures = 5000 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = 01e1476b-1467-4e0e-b01c-0f8cdf8276fe 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout:[osd] 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: osd journal size = 100 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub load threshold = 5.0 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: osd scrub max interval = 600 2024-09-13T05:01:11.051 INFO:teuthology.orchestra.run.smithi110.stdout: osd mclock profile = high_recovery_ops 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd recover clone overlap = true 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd recovery max chunk = 1048576 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug shutdown = true 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug op order = true 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug verify stray on activate = true 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd open classes on start = true 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug pg log writeout = true 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd deep scrub update digest min age = 30 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.052 INFO:teuthology.orchestra.run.smithi110.stdout: osd map max advance = 10 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: journal zero on create = true 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: filestore ondisk finisher threads = 3 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: filestore apply finisher threads = 3 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: bdev debug aio = true 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: osd debug misdirected ops = true 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore block size = 96636764160 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore compression algorithm = zstd 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore compression mode = aggressive 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: bluestore fsck on mount = True 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluefs = 1/20 2024-09-13T05:01:11.053 INFO:teuthology.orchestra.run.smithi110.stdout: debug bluestore = 1/20 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: debug osd = 20 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: debug rocksdb = 4/10 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd backfillfull_ratio = 0.85 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd full ratio = 0.9 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd nearfull ratio = 0.8 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: osd blocked scrub grace period = 3600 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: osd failsafe full ratio = 0.95 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: osd max markdown count = 1000 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: osd objectstore = bluestore 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: osd shutdown pgref assert = True 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.054 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr] 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug mgr = 20 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: mgr/telemetry/nag = false 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout:[mon] 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug ms = 1 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug mon = 20 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug paxos = 20 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: debug auth = 20 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: mon data avail warn = 5 2024-09-13T05:01:11.055 INFO:teuthology.orchestra.run.smithi110.stdout: mon mgr mkfs grace = 240 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min pgs per osd = 4 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd reporter subtree level = osd 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: mon osd prime pg temp = true 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: mon reweight min bytes per osd = 10 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: auth mon ticket ttl = 660 # 11m 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: auth service ticket ttl = 240 # 4m 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: # don't complain about insecure global_id in the test suite 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-13T05:01:11.056 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: # 1m isn't quite enough 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: mon_down_mkfs_grace = 2m 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: mon_warn_on_filestore_osds = false 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout:[client] 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: rgw cache enabled = true 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable ops log = true 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: rgw enable usage log = true 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: rbd cache = True 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout: rbd cache max dirty = 0 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.a] 2024-09-13T05:01:11.057 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.c] 2024-09-13T05:01:11.058 INFO:teuthology.orchestra.run.smithi110.stdout:[mon.b] 2024-09-13T05:01:11.058 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-13T05:01:11.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-13T05:01:11.138 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-13T05:01:11.138 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:11.138 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-13T05:01:11.154 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:11.154 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-13T05:01:11.210 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:01:11.210 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:11.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T05:01:11.210 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T05:01:11.286 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:11.286 DEBUG:teuthology.orchestra.run.smithi053:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:11.342 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:01:11.342 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:11.342 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-13T05:01:11.342 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-13T05:01:11.377 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:11.377 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:11.433 INFO:tasks.ceph:Setting up mon nodes... 2024-09-13T05:01:11.433 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-13T05:01:11.433 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T05:01:11.486 INFO:teuthology.orchestra.run.smithi053.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-13T05:01:11.488 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T05:01:11.549 INFO:teuthology.orchestra.run.smithi110.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-13T05:01:11.551 INFO:tasks.ceph:Setting up mds nodes... 2024-09-13T05:01:11.552 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-13T05:01:11.552 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-13T05:01:11.595 INFO:teuthology.orchestra.run.smithi053.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-13T05:01:11.608 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-13T05:01:11.648 INFO:teuthology.orchestra.run.smithi110.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-13T05:01:11.661 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-13T05:01:11.661 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi053.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi110.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-13T05:01:11.661 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-13T05:01:11.686 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:11.687 INFO:tasks.ceph:role: osd.0 2024-09-13T05:01:11.687 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:01:11.687 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T05:01:11.751 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:11.752 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:11.783 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-13T05:01:11.785 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-13T05:01:11.785 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-13T05:01:11.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-13T05:01:11.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-13T05:01:11.992 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:11.992 INFO:tasks.ceph:role: osd.1 2024-09-13T05:01:11.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:01:11.993 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T05:01:12.057 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:12.058 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:12.090 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-13T05:01:12.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-13T05:01:12.093 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-13T05:01:12.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-13T05:01:12.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-13T05:01:12.298 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:12.299 INFO:tasks.ceph:role: osd.2 2024-09-13T05:01:12.299 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:01:12.299 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:12.364 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:12.396 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-13T05:01:12.398 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-13T05:01:12.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-13T05:01:12.468 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-13T05:01:12.537 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-13T05:01:12.602 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:12.603 INFO:tasks.ceph:role: osd.3 2024-09-13T05:01:12.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:01:12.603 DEBUG:teuthology.orchestra.run.smithi053:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:12.668 INFO:teuthology.orchestra.run.smithi053.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:12.669 INFO:teuthology.orchestra.run.smithi053.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:12.669 INFO:teuthology.orchestra.run.smithi053.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:12.704 INFO:teuthology.orchestra.run.smithi053.stdout:Discarding blocks...Done. 2024-09-13T05:01:12.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi053.front.sepia.ceph.com -o noatime 2024-09-13T05:01:12.706 DEBUG:teuthology.orchestra.run.smithi053:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-13T05:01:12.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-13T05:01:12.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:12.960+0000 7f3c697c5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-13T05:01:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:12.960+0000 7f3c697c5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-13T05:01:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:12.960+0000 7f3c697c5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T05:01:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:12.960+0000 7f3c697c5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T05:01:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:12.960+0000 7f3c697c5640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-13T05:01:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:12.960+0000 7f3c697c5640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-13T05:01:15.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-13T05:01:15.846 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:15.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:15.937+0000 7f847ba0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-13T05:01:15.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:15.937+0000 7f847ba0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-13T05:01:15.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:15.937+0000 7f847ba0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T05:01:15.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:15.937+0000 7f847ba0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T05:01:15.938 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:15.937+0000 7f847ba0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-13T05:01:15.939 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:15.937+0000 7f847ba0d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-13T05:01:18.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-13T05:01:18.820 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:18.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:18.913+0000 7fd627541640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-13T05:01:18.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:18.913+0000 7fd627541640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-13T05:01:18.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:18.913+0000 7fd627541640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T05:01:18.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:18.913+0000 7fd627541640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T05:01:18.914 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:18.913+0000 7fd627541640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-13T05:01:18.915 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:18.913+0000 7fd627541640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-13T05:01:21.766 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-13T05:01:21.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:21.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:21.891+0000 7fd3f0416640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-13T05:01:21.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:21.891+0000 7fd3f0416640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-13T05:01:21.892 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:21.891+0000 7fd3f0416640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T05:01:21.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:21.891+0000 7fd3f0416640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T05:01:21.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:21.891+0000 7fd3f0416640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-13T05:01:21.893 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:01:21.891+0000 7fd3f0416640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-13T05:01:24.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-13T05:01:24.774 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-13T05:01:24.805 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:24.805 INFO:tasks.ceph:role: osd.4 2024-09-13T05:01:24.805 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:01:24.805 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-13T05:01:24.873 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:24.874 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:24.912 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-13T05:01:24.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-13T05:01:24.914 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-13T05:01:24.986 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-13T05:01:25.055 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-13T05:01:25.122 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:25.122 INFO:tasks.ceph:role: osd.5 2024-09-13T05:01:25.122 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:01:25.122 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-13T05:01:25.186 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:25.186 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:25.186 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:25.186 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:25.186 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:25.186 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:25.187 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:25.187 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:25.187 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:25.187 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:25.219 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-13T05:01:25.221 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-13T05:01:25.221 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-13T05:01:25.292 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-13T05:01:25.360 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-13T05:01:25.425 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:25.426 INFO:tasks.ceph:role: osd.6 2024-09-13T05:01:25.426 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:01:25.426 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-13T05:01:25.491 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:25.492 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:25.524 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-13T05:01:25.527 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-13T05:01:25.527 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-13T05:01:25.596 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-13T05:01:25.666 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-13T05:01:25.732 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-13T05:01:25.732 INFO:tasks.ceph:role: osd.7 2024-09-13T05:01:25.732 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:01:25.732 DEBUG:teuthology.orchestra.run.smithi110:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-13T05:01:25.796 INFO:teuthology.orchestra.run.smithi110.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-13T05:01:25.796 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-13T05:01:25.796 INFO:teuthology.orchestra.run.smithi110.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-13T05:01:25.796 INFO:teuthology.orchestra.run.smithi110.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-13T05:01:25.797 INFO:teuthology.orchestra.run.smithi110.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-13T05:01:25.797 INFO:teuthology.orchestra.run.smithi110.stdout: = sunit=0 swidth=0 blks 2024-09-13T05:01:25.797 INFO:teuthology.orchestra.run.smithi110.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-13T05:01:25.797 INFO:teuthology.orchestra.run.smithi110.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-13T05:01:25.797 INFO:teuthology.orchestra.run.smithi110.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-13T05:01:25.797 INFO:teuthology.orchestra.run.smithi110.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-13T05:01:25.828 INFO:teuthology.orchestra.run.smithi110.stdout:Discarding blocks...Done. 2024-09-13T05:01:25.830 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi110.front.sepia.ceph.com -o noatime 2024-09-13T05:01:25.830 DEBUG:teuthology.orchestra.run.smithi110:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-13T05:01:25.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-13T05:01:25.968 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:26.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:26.064+0000 7f27ba80e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-13T05:01:26.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:26.064+0000 7f27ba80e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-13T05:01:26.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:26.064+0000 7f27ba80e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T05:01:26.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:26.064+0000 7f27ba80e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T05:01:26.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:26.064+0000 7f27ba80e640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-13T05:01:26.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:26.065+0000 7f27ba80e640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-13T05:01:28.909 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-13T05:01:28.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:29.026+0000 7f58d4224640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-13T05:01:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:29.027+0000 7f58d4224640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-13T05:01:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:29.027+0000 7f58d4224640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T05:01:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:29.027+0000 7f58d4224640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T05:01:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:29.027+0000 7f58d4224640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-13T05:01:29.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:29.027+0000 7f58d4224640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-13T05:01:31.876 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-13T05:01:31.903 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:31.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:31.989+0000 7f07ff826640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-13T05:01:31.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:31.989+0000 7f07ff826640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-13T05:01:31.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:31.989+0000 7f07ff826640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T05:01:31.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:31.989+0000 7f07ff826640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T05:01:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:31.989+0000 7f07ff826640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-13T05:01:31.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:31.989+0000 7f07ff826640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-13T05:01:34.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-13T05:01:34.856 DEBUG:teuthology.orchestra.run.smithi110:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:34.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:34.943+0000 7f5d37c19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-13T05:01:34.944 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:34.943+0000 7f5d37c19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-13T05:01:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:34.943+0000 7f5d37c19640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T05:01:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:34.943+0000 7f5d37c19640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T05:01:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:34.943+0000 7f5d37c19640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-13T05:01:34.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-09-13T05:01:34.943+0000 7f5d37c19640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-13T05:01:37.790 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-13T05:01:37.819 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-13T05:01:37.819 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:37.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-13T05:01:37.851 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:37.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-13T05:01:37.922 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:37.923 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-13T05:01:37.992 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:37.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-13T05:01:38.061 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:38.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-13T05:01:38.132 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:38.132 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-13T05:01:38.162 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:38.162 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-13T05:01:38.231 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:38.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-13T05:01:38.298 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:38.299 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-13T05:01:38.365 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:38.365 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-13T05:01:38.431 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:38.431 DEBUG:teuthology.orchestra.run.smithi053:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-13T05:01:38.450 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:38.450 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-13T05:01:38.487 INFO:tasks.ceph:Adding keys to all mons... 2024-09-13T05:01:38.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T05:01:38.493 DEBUG:teuthology.orchestra.run.smithi110:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-13T05:01:38.530 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.y] 2024-09-13T05:01:38.530 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXx+NmdRf/HBAAcPI0B4AQirhzzrxR6Fop7A== 2024-09-13T05:01:38.530 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.0] 2024-09-13T05:01:38.530 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAYx+Nm+I5FORAA03d8S8SBRjf3lWIX6CkxrA== 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.1] 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAbx+NmstzrNxAAR9k5VhnfVX6EL1mboZLmag== 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.2] 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAex+NmNuB5NhAA0perywhc6DgJEw1IbpxlxQ== 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.3] 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAhx+NmRPArNRAAW4Usm/9o2KfFDuFXr517lQ== 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:[mgr.x] 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXx+NmZxPDIBAABTZAyTd4ghLoj3CuQQ2yQQ== 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.4] 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAmx+Nm2j3mAxAAPXJKzeMnyBb3lKeMae/Q+g== 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.5] 2024-09-13T05:01:38.531 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQApx+NmjCOoARAASwJVlxsJw3/rvdU1H4EvRg== 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.6] 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQArx+NmeG4DOxAAqmIxdOj9P+AR6E+GlQe1XQ== 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout:[osd.7] 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAux+NmEbtJOBAAW3xBAKJG1LODlf7o1uvDmw== 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout:[client.0] 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXx+Nm1tZ5IxAAMrE/4l+M2KRyXT2uFgKLlg== 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout:[client.1] 2024-09-13T05:01:38.532 INFO:teuthology.orchestra.run.smithi053.stdout: key = AQAXx+NmL5StJhAAFLujsF+GwpZXkbPzj8RpEA== 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.y] 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAXx+NmdRf/HBAAcPI0B4AQirhzzrxR6Fop7A== 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.0] 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAYx+Nm+I5FORAA03d8S8SBRjf3lWIX6CkxrA== 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.1] 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAbx+NmstzrNxAAR9k5VhnfVX6EL1mboZLmag== 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.2] 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAex+NmNuB5NhAA0perywhc6DgJEw1IbpxlxQ== 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.3] 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAhx+NmRPArNRAAW4Usm/9o2KfFDuFXr517lQ== 2024-09-13T05:01:38.551 INFO:teuthology.orchestra.run.smithi110.stdout:[mgr.x] 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAXx+NmZxPDIBAABTZAyTd4ghLoj3CuQQ2yQQ== 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.4] 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAmx+Nm2j3mAxAAPXJKzeMnyBb3lKeMae/Q+g== 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.5] 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQApx+NmjCOoARAASwJVlxsJw3/rvdU1H4EvRg== 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.6] 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQArx+NmeG4DOxAAqmIxdOj9P+AR6E+GlQe1XQ== 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout:[osd.7] 2024-09-13T05:01:38.552 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAux+NmEbtJOBAAW3xBAKJG1LODlf7o1uvDmw== 2024-09-13T05:01:38.553 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-09-13T05:01:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAXx+Nm1tZ5IxAAMrE/4l+M2KRyXT2uFgKLlg== 2024-09-13T05:01:38.553 INFO:teuthology.orchestra.run.smithi110.stdout:[client.1] 2024-09-13T05:01:38.553 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQAXx+NmL5StJhAAFLujsF+GwpZXkbPzj8RpEA== 2024-09-13T05:01:38.554 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T05:01:38.576 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T05:01:38.644 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.646 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.698 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.700 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.755 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T05:01:38.859 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-13T05:01:38.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.912 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.963 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:38.965 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:39.015 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:39.017 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:39.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:39.070 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-13T05:01:39.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T05:01:39.127 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T05:01:39.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T05:01:39.183 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-13T05:01:39.236 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-13T05:01:39.236 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-13T05:01:39.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T05:01:39.359 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-13T05:01:39.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-13T05:01:39.450 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T05:01:39.548 DEBUG:teuthology.orchestra.run.smithi053:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-13T05:01:39.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-13T05:01:39.599 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-13T05:01:39.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-13T05:01:39.717 DEBUG:teuthology.orchestra.run.smithi053:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:39.720 DEBUG:teuthology.orchestra.run.smithi110:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-13T05:01:39.773 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-13T05:01:39.773 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-13T05:01:39.773 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-13T05:01:39.776 INFO:tasks.ceph.mon.a:Started 2024-09-13T05:01:39.777 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-13T05:01:39.777 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-13T05:01:39.779 INFO:tasks.ceph.mon.c:Started 2024-09-13T05:01:39.779 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-13T05:01:39.779 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-13T05:01:39.815 INFO:tasks.ceph.mon.b:Started 2024-09-13T05:01:39.815 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-13T05:01:39.815 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-13T05:01:39.816 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-13T05:01:39.818 INFO:tasks.ceph.mgr.y:Started 2024-09-13T05:01:39.818 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-13T05:01:39.818 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-13T05:01:39.820 INFO:tasks.ceph.mgr.x:Started 2024-09-13T05:01:39.821 INFO:tasks.ceph:Setting crush tunables to default 2024-09-13T05:01:39.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd crush tunables default 2024-09-13T05:01:45.166 INFO:teuthology.orchestra.run.smithi053.stderr:adjusted tunables profile to default 2024-09-13T05:01:45.179 INFO:tasks.ceph:check_enable_crimson: False 2024-09-13T05:01:45.179 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-13T05:01:45.179 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:45.179 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-13T05:01:45.206 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:45.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-13T05:01:45.273 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:45.273 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-13T05:01:45.339 DEBUG:teuthology.orchestra.run.smithi053:> set -ex 2024-09-13T05:01:45.339 DEBUG:teuthology.orchestra.run.smithi053:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-13T05:01:45.405 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:45.406 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-13T05:01:45.436 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:45.436 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-13T05:01:45.504 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:45.504 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-13T05:01:45.572 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-09-13T05:01:45.572 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-13T05:01:45.639 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new baf16a5c-4204-424b-8d8e-a5b1486aa052 0 2024-09-13T05:01:45.857 INFO:teuthology.orchestra.run.smithi110.stdout:0 2024-09-13T05:01:45.867 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new f3df9a0f-3a9e-4665-9233-718bc0fe1223 1 2024-09-13T05:01:45.995 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:45.994+0000 7f3252b23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T05:01:46.014 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.014+0000 7f7669d23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-13T05:01:46.036 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.035+0000 7f3252b23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T05:01:46.050 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-09-13T05:01:46.060 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.059+0000 7f7669d23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-13T05:01:46.061 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 1304a876-75c5-454b-a60b-8c9823c4aca8 2 2024-09-13T05:01:46.189 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.189+0000 7f3252b23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T05:01:46.217 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.217+0000 7f7669d23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-13T05:01:46.238 INFO:teuthology.orchestra.run.smithi110.stdout:2 2024-09-13T05:01:46.241 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.240+0000 7f3252b23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T05:01:46.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 83668012-8ddb-4fe5-bb31-6e78d2723f12 3 2024-09-13T05:01:46.270 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.269+0000 7f7669d23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-13T05:01:46.411 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.410+0000 7f3252b23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T05:01:46.429 INFO:teuthology.orchestra.run.smithi110.stdout:3 2024-09-13T05:01:46.439 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.438+0000 7f7669d23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-13T05:01:46.440 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new f462232f-5144-4652-9139-9039d53e191a 4 2024-09-13T05:01:46.527 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.526+0000 7f3252b23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T05:01:46.554 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.553+0000 7f7669d23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-13T05:01:46.617 INFO:teuthology.orchestra.run.smithi110.stdout:4 2024-09-13T05:01:46.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 1d2d8607-f1d9-45e3-8486-3f5bb23b516b 5 2024-09-13T05:01:46.804 INFO:teuthology.orchestra.run.smithi110.stdout:5 2024-09-13T05:01:46.815 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 7fe0fb7d-0a40-4666-82a1-4e18f53b445c 6 2024-09-13T05:01:46.872 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.872+0000 7f3252b23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T05:01:46.875 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.874+0000 7f7669d23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-13T05:01:46.930 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:46.930+0000 7f7669d23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T05:01:46.936 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:46.935+0000 7f3252b23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-13T05:01:46.991 INFO:teuthology.orchestra.run.smithi110.stdout:6 2024-09-13T05:01:47.004 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --cluster ceph osd new 811503be-abe3-4d64-a094-09ff675f3666 7 2024-09-13T05:01:47.085 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:47.084+0000 7f7669d23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T05:01:47.090 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:47.090+0000 7f3252b23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-13T05:01:47.181 INFO:teuthology.orchestra.run.smithi110.stdout:7 2024-09-13T05:01:47.193 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-13T05:01:47.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-13T05:01:47.196 INFO:tasks.ceph.osd.0:Started 2024-09-13T05:01:47.196 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-13T05:01:47.196 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-13T05:01:47.199 INFO:tasks.ceph.osd.1:Started 2024-09-13T05:01:47.199 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-13T05:01:47.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-13T05:01:47.201 INFO:tasks.ceph.osd.2:Started 2024-09-13T05:01:47.201 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-13T05:01:47.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-13T05:01:47.203 INFO:tasks.ceph.osd.3:Started 2024-09-13T05:01:47.203 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-13T05:01:47.204 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-13T05:01:47.206 INFO:tasks.ceph.osd.4:Started 2024-09-13T05:01:47.206 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-13T05:01:47.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-13T05:01:47.208 INFO:tasks.ceph.osd.5:Started 2024-09-13T05:01:47.208 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-13T05:01:47.208 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-13T05:01:47.210 INFO:tasks.ceph.osd.6:Started 2024-09-13T05:01:47.211 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-13T05:01:47.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-13T05:01:47.213 INFO:tasks.ceph.osd.7:Started 2024-09-13T05:01:47.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:01:47.408 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:47.408 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":10,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:01:47.179750+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T05:01:47.179750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:01:47.420 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-13T05:01:47.420 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-13T05:01:47.805 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:01:47.804+0000 7fdb67558640 -1 Falling back to public interface 2024-09-13T05:01:47.807 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:01:47.806+0000 7f6118616640 -1 Falling back to public interface 2024-09-13T05:01:47.808 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:01:47.807+0000 7f95bcf34640 -1 Falling back to public interface 2024-09-13T05:01:47.809 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:01:47.808+0000 7f95bbedd640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6816/0: (98) Address already in use 2024-09-13T05:01:47.817 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:01:47.816+0000 7fb491f2c640 -1 Falling back to public interface 2024-09-13T05:01:47.821 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:01:47.820+0000 7f4962b58640 -1 Falling back to public interface 2024-09-13T05:01:47.825 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:01:47.824+0000 7f0a9d20c640 -1 Falling back to public interface 2024-09-13T05:01:47.828 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:01:47.828+0000 7fc95d8e2640 -1 Falling back to public interface 2024-09-13T05:01:47.829 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:01:47.828+0000 7f70e872c640 -1 Falling back to public interface 2024-09-13T05:01:48.067 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.066+0000 7f3252b23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T05:01:48.120 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.119+0000 7f3252b23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T05:01:48.132 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.131+0000 7f7669d23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-13T05:01:48.186 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.184+0000 7f7669d23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-13T05:01:48.226 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T05:01:48.387 INFO:teuthology.misc.health.smithi053.stdout: 2024-09-13T05:01:48.387 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":10,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:01:47.179750+0000","last_up_change":"0.000000","last_in_change":"2024-09-13T05:01:47.179750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:01:48.397 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-13T05:01:48.486 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.486+0000 7f3252b23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T05:01:48.571 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.571+0000 7f3252b23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T05:01:48.573 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.572+0000 7f7669d23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-13T05:01:48.662 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.660+0000 7f7669d23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-13T05:01:48.678 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.678+0000 7f3252b23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T05:01:48.724 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.724+0000 7f3252b23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T05:01:48.771 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.770+0000 7f7669d23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-13T05:01:48.819 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.817+0000 7f7669d23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-13T05:01:48.835 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.834+0000 7f3252b23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T05:01:48.889 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.888+0000 7f3252b23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T05:01:48.934 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:48.933+0000 7f3252b23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T05:01:48.935 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.933+0000 7f7669d23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-13T05:01:48.990 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:48.988+0000 7f7669d23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-13T05:01:49.037 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.035+0000 7f7669d23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-13T05:01:49.042 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.042+0000 7f3252b23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T05:01:49.092 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.091+0000 7f3252b23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T05:01:49.142 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.142+0000 7f3252b23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T05:01:49.152 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.151+0000 7f7669d23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-13T05:01:49.205 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.203+0000 7f7669d23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-13T05:01:49.211 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.210+0000 7f3252b23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T05:01:49.255 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.254+0000 7f3252b23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T05:01:49.258 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.256+0000 7f7669d23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-13T05:01:49.298 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.297+0000 7f3252b23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T05:01:49.328 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.326+0000 7f7669d23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-13T05:01:49.374 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.372+0000 7f7669d23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-13T05:01:49.419 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:49.418+0000 7f7669d23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-13T05:01:49.931 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:49.930+0000 7f3252b23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T05:01:50.095 INFO:tasks.ceph.mgr.y.smithi053.stderr:2024-09-13T05:01:50.093+0000 7f7669d23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-13T05:01:50.945 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:50.944+0000 7f31f3528640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-13T05:01:50.945 INFO:tasks.ceph.mgr.x.smithi110.stderr:2024-09-13T05:01:50.944+0000 7f31f3528640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-13T05:01:51.292 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:01:51.291+0000 7f6118616640 -1 osd.4 0 log_to_monitors true 2024-09-13T05:01:51.315 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:01:51.313+0000 7f0a9d20c640 -1 osd.0 0 log_to_monitors true 2024-09-13T05:01:51.328 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:01:51.327+0000 7f95bcf34640 -1 osd.6 0 log_to_monitors true 2024-09-13T05:01:51.328 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:01:51.328+0000 7fdb67558640 -1 osd.5 0 log_to_monitors true 2024-09-13T05:01:51.349 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:01:51.347+0000 7f4962b58640 -1 osd.2 0 log_to_monitors true 2024-09-13T05:01:51.349 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:01:51.348+0000 7f70e872c640 -1 osd.3 0 log_to_monitors true 2024-09-13T05:01:51.355 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:01:51.354+0000 7fc95d8e2640 -1 osd.7 0 log_to_monitors true 2024-09-13T05:01:51.360 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:01:51.358+0000 7fb491f2c640 -1 osd.1 0 log_to_monitors true 2024-09-13T05:01:54.158 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:01:54.157+0000 7fdb634e5640 -1 osd.5 0 waiting for initial osdmap 2024-09-13T05:01:54.170 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:01:54.169+0000 7fdb5b306640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.172 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:01:54.171+0000 7f95b96d8640 -1 osd.6 0 waiting for initial osdmap 2024-09-13T05:01:54.185 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:01:54.185+0000 7f95b0ce2640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.192 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:01:54.191+0000 7f6113e9c640 -1 osd.4 0 waiting for initial osdmap 2024-09-13T05:01:54.201 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:01:54.201+0000 7f610bcbd640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.206 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:01:54.206+0000 7fc95a086640 -1 osd.7 0 waiting for initial osdmap 2024-09-13T05:01:54.216 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:01:54.216+0000 7fc951690640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.263 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:01:54.262+0000 7f0a98a84640 -1 osd.0 0 waiting for initial osdmap 2024-09-13T05:01:54.278 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:01:54.277+0000 7f0a908a5640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.293 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:01:54.292+0000 7f495eae5640 -1 osd.2 0 waiting for initial osdmap 2024-09-13T05:01:54.302 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:01:54.300+0000 7fb48deb9640 -1 osd.1 0 waiting for initial osdmap 2024-09-13T05:01:54.306 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:01:54.305+0000 7f4956906640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.316 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:01:54.315+0000 7fb485cda640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:54.319 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:01:54.318+0000 7f70e46b9640 -1 osd.3 0 waiting for initial osdmap 2024-09-13T05:01:54.334 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:01:54.333+0000 7f70dc4da640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-13T05:01:55.204 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-13T05:01:55.493 INFO:teuthology.misc.health.smithi053.stdout: 2024-09-13T05:01:55.493 INFO:teuthology.misc.health.smithi053.stdout:{"epoch":13,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:01:55.160352+0000","last_up_change":"2024-09-13T05:01:55.160352+0000","last_in_change":"2024-09-13T05:01:47.179750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6817","nonce":1769628429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6819","nonce":1769628429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6823","nonce":1769628429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6821","nonce":1769628429}]},"public_addr":"172.21.15.53:6817/1769628429","cluster_addr":"172.21.15.53:6819/1769628429","heartbeat_back_addr":"172.21.15.53:6823/1769628429","heartbeat_front_addr":"172.21.15.53:6821/1769628429","state":["exists","up"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6801","nonce":827277153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6803","nonce":827277153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6807","nonce":827277153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6805","nonce":827277153}]},"public_addr":"172.21.15.53:6801/827277153","cluster_addr":"172.21.15.53:6803/827277153","heartbeat_back_addr":"172.21.15.53:6807/827277153","heartbeat_front_addr":"172.21.15.53:6805/827277153","state":["exists","up"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6809","nonce":1334123504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6811","nonce":1334123504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6815","nonce":1334123504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6813","nonce":1334123504}]},"public_addr":"172.21.15.53:6809/1334123504","cluster_addr":"172.21.15.53:6811/1334123504","heartbeat_back_addr":"172.21.15.53:6815/1334123504","heartbeat_front_addr":"172.21.15.53:6813/1334123504","state":["exists","up"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6825","nonce":4134389313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6827","nonce":4134389313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6831","nonce":4134389313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6829","nonce":4134389313}]},"public_addr":"172.21.15.53:6825/4134389313","cluster_addr":"172.21.15.53:6827/4134389313","heartbeat_back_addr":"172.21.15.53:6831/4134389313","heartbeat_front_addr":"172.21.15.53:6829/4134389313","state":["exists","up"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6809","nonce":3681797816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6811","nonce":3681797816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6815","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6816","nonce":3681797816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6813","nonce":3681797816}]},"public_addr":"172.21.15.110:6809/3681797816","cluster_addr":"172.21.15.110:6811/3681797816","heartbeat_back_addr":"172.21.15.110:6816/3681797816","heartbeat_front_addr":"172.21.15.110:6813/3681797816","state":["exists","up"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6801","nonce":81482652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6803","nonce":81482652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6807","nonce":81482652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6805","nonce":81482652}]},"public_addr":"172.21.15.110:6801/81482652","cluster_addr":"172.21.15.110:6803/81482652","heartbeat_back_addr":"172.21.15.110:6807/81482652","heartbeat_front_addr":"172.21.15.110:6805/81482652","state":["exists","up"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6817","nonce":576600432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6819","nonce":576600432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6823","nonce":576600432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6821","nonce":576600432}]},"public_addr":"172.21.15.110:6817/576600432","cluster_addr":"172.21.15.110:6819/576600432","heartbeat_back_addr":"172.21.15.110:6823/576600432","heartbeat_front_addr":"172.21.15.110:6821/576600432","state":["exists","up"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6825","nonce":896779674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6827","nonce":896779674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6831","nonce":896779674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6829","nonce":896779674}]},"public_addr":"172.21.15.110:6825/896779674","cluster_addr":"172.21.15.110:6827/896779674","heartbeat_back_addr":"172.21.15.110:6831/896779674","heartbeat_front_addr":"172.21.15.110:6829/896779674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:01:55.504 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-13T05:01:55.504 INFO:tasks.ceph:Creating RBD pool 2024-09-13T05:01:55.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-13T05:01:56.168 INFO:teuthology.orchestra.run.smithi053.stderr:pool 'rbd' created 2024-09-13T05:01:56.182 DEBUG:teuthology.orchestra.run.smithi053:> rbd --cluster ceph pool init rbd 2024-09-13T05:01:59.202 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-13T05:01:59.203 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-13T05:01:59.203 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-13T05:01:59.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-13T05:01:59.204 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-13T05:01:59.549 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:59.559 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2133598560},{"type":"v1","addr":"172.21.15.110:6833","nonce":2133598560}]},"active_addr":"172.21.15.110:6833/2133598560","active_change":"2024-09-13T05:01:49.933539+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4118,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3012389072}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":4031988898}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3988434124}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":4255272143}]}]} 2024-09-13T05:01:59.564 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-13T05:01:59.564 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T05:01:59.565 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:01:59.850 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:01:59.851 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":17,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:01:59.192143+0000","last_up_change":"2024-09-13T05:01:55.160352+0000","last_in_change":"2024-09-13T05:01:47.179750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T05:01:55.817858+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T05:01:55.981425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6817","nonce":1769628429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6819","nonce":1769628429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6823","nonce":1769628429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6821","nonce":1769628429}]},"public_addr":"172.21.15.53:6817/1769628429","cluster_addr":"172.21.15.53:6819/1769628429","heartbeat_back_addr":"172.21.15.53:6823/1769628429","heartbeat_front_addr":"172.21.15.53:6821/1769628429","state":["exists","up"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6801","nonce":827277153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6803","nonce":827277153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6807","nonce":827277153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6805","nonce":827277153}]},"public_addr":"172.21.15.53:6801/827277153","cluster_addr":"172.21.15.53:6803/827277153","heartbeat_back_addr":"172.21.15.53:6807/827277153","heartbeat_front_addr":"172.21.15.53:6805/827277153","state":["exists","up"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6809","nonce":1334123504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6811","nonce":1334123504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6815","nonce":1334123504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6813","nonce":1334123504}]},"public_addr":"172.21.15.53:6809/1334123504","cluster_addr":"172.21.15.53:6811/1334123504","heartbeat_back_addr":"172.21.15.53:6815/1334123504","heartbeat_front_addr":"172.21.15.53:6813/1334123504","state":["exists","up"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6825","nonce":4134389313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6827","nonce":4134389313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6831","nonce":4134389313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6829","nonce":4134389313}]},"public_addr":"172.21.15.53:6825/4134389313","cluster_addr":"172.21.15.53:6827/4134389313","heartbeat_back_addr":"172.21.15.53:6831/4134389313","heartbeat_front_addr":"172.21.15.53:6829/4134389313","state":["exists","up"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6809","nonce":3681797816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6811","nonce":3681797816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6815","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6816","nonce":3681797816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6813","nonce":3681797816}]},"public_addr":"172.21.15.110:6809/3681797816","cluster_addr":"172.21.15.110:6811/3681797816","heartbeat_back_addr":"172.21.15.110:6816/3681797816","heartbeat_front_addr":"172.21.15.110:6813/3681797816","state":["exists","up"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6801","nonce":81482652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6803","nonce":81482652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6807","nonce":81482652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6805","nonce":81482652}]},"public_addr":"172.21.15.110:6801/81482652","cluster_addr":"172.21.15.110:6803/81482652","heartbeat_back_addr":"172.21.15.110:6807/81482652","heartbeat_front_addr":"172.21.15.110:6805/81482652","state":["exists","up"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6817","nonce":576600432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6819","nonce":576600432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6823","nonce":576600432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6821","nonce":576600432}]},"public_addr":"172.21.15.110:6817/576600432","cluster_addr":"172.21.15.110:6819/576600432","heartbeat_back_addr":"172.21.15.110:6823/576600432","heartbeat_front_addr":"172.21.15.110:6821/576600432","state":["exists","up"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6825","nonce":896779674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6827","nonce":896779674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6831","nonce":896779674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6829","nonce":896779674}]},"public_addr":"172.21.15.110:6825/896779674","cluster_addr":"172.21.15.110:6827/896779674","heartbeat_back_addr":"172.21.15.110:6831/896779674","heartbeat_front_addr":"172.21.15.110:6829/896779674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358655+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.343203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.307081+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.315153+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.280902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.302500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.354685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:01:59.863 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T05:01:59.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:02:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:00.149 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":17,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:01:59.192143+0000","last_up_change":"2024-09-13T05:01:55.160352+0000","last_in_change":"2024-09-13T05:01:47.179750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T05:01:55.817858+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T05:01:55.981425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6817","nonce":1769628429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6819","nonce":1769628429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6823","nonce":1769628429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6821","nonce":1769628429}]},"public_addr":"172.21.15.53:6817/1769628429","cluster_addr":"172.21.15.53:6819/1769628429","heartbeat_back_addr":"172.21.15.53:6823/1769628429","heartbeat_front_addr":"172.21.15.53:6821/1769628429","state":["exists","up"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6801","nonce":827277153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6803","nonce":827277153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6807","nonce":827277153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6805","nonce":827277153}]},"public_addr":"172.21.15.53:6801/827277153","cluster_addr":"172.21.15.53:6803/827277153","heartbeat_back_addr":"172.21.15.53:6807/827277153","heartbeat_front_addr":"172.21.15.53:6805/827277153","state":["exists","up"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6809","nonce":1334123504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6811","nonce":1334123504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6815","nonce":1334123504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6813","nonce":1334123504}]},"public_addr":"172.21.15.53:6809/1334123504","cluster_addr":"172.21.15.53:6811/1334123504","heartbeat_back_addr":"172.21.15.53:6815/1334123504","heartbeat_front_addr":"172.21.15.53:6813/1334123504","state":["exists","up"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6825","nonce":4134389313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6827","nonce":4134389313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6831","nonce":4134389313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6829","nonce":4134389313}]},"public_addr":"172.21.15.53:6825/4134389313","cluster_addr":"172.21.15.53:6827/4134389313","heartbeat_back_addr":"172.21.15.53:6831/4134389313","heartbeat_front_addr":"172.21.15.53:6829/4134389313","state":["exists","up"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6809","nonce":3681797816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6811","nonce":3681797816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6815","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6816","nonce":3681797816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6813","nonce":3681797816}]},"public_addr":"172.21.15.110:6809/3681797816","cluster_addr":"172.21.15.110:6811/3681797816","heartbeat_back_addr":"172.21.15.110:6816/3681797816","heartbeat_front_addr":"172.21.15.110:6813/3681797816","state":["exists","up"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6801","nonce":81482652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6803","nonce":81482652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6807","nonce":81482652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6805","nonce":81482652}]},"public_addr":"172.21.15.110:6801/81482652","cluster_addr":"172.21.15.110:6803/81482652","heartbeat_back_addr":"172.21.15.110:6807/81482652","heartbeat_front_addr":"172.21.15.110:6805/81482652","state":["exists","up"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6817","nonce":576600432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6819","nonce":576600432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6823","nonce":576600432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6821","nonce":576600432}]},"public_addr":"172.21.15.110:6817/576600432","cluster_addr":"172.21.15.110:6819/576600432","heartbeat_back_addr":"172.21.15.110:6823/576600432","heartbeat_front_addr":"172.21.15.110:6821/576600432","state":["exists","up"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6825","nonce":896779674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6827","nonce":896779674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6831","nonce":896779674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6829","nonce":896779674}]},"public_addr":"172.21.15.110:6825/896779674","cluster_addr":"172.21.15.110:6827/896779674","heartbeat_back_addr":"172.21.15.110:6831/896779674","heartbeat_front_addr":"172.21.15.110:6829/896779674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358655+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.343203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.307081+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.315153+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.280902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.302500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.354685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:02:00.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T05:02:00.162 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T05:02:00.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T05:02:00.163 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T05:02:00.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T05:02:00.164 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T05:02:00.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T05:02:00.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T05:02:00.414 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:00.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T05:02:00.418 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.418 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.419 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T05:02:00.419 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T05:02:00.420 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T05:02:00.421 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.422 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T05:02:00.423 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.423 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T05:02:00.424 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T05:02:00.425 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.426 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T05:02:00.920 INFO:teuthology.orchestra.run.smithi053.stdout:55834574851 2024-09-13T05:02:00.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2024-09-13T05:02:01.007 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-09-13T05:02:01.045 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.128 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.129 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-09-13T05:02:01.130 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-09-13T05:02:01.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2024-09-13T05:02:01.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-09-13T05:02:01.132 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.132 INFO:teuthology.orchestra.run.smithi053.stdout:55834574850 2024-09-13T05:02:01.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-09-13T05:02:01.134 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2024-09-13T05:02:01.940 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T05:02:02.020 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T05:02:02.130 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T05:02:02.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T05:02:02.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T05:02:02.132 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T05:02:02.133 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T05:02:02.135 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T05:02:02.376 INFO:teuthology.orchestra.run.smithi053.stdout:55834574853 2024-09-13T05:02:02.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.1 2024-09-13T05:02:02.394 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.519 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.6 2024-09-13T05:02:02.538 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.542 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.5 2024-09-13T05:02:02.561 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.618 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.3 2024-09-13T05:02:02.631 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.697 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.698 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.7 2024-09-13T05:02:02.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.0 2024-09-13T05:02:02.713 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.713 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.717 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.4 2024-09-13T05:02:02.735 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.749 INFO:teuthology.orchestra.run.smithi053.stdout:55834574852 2024-09-13T05:02:02.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.2 2024-09-13T05:02:02.761 DEBUG:teuthology.parallel:result is None 2024-09-13T05:02:02.761 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T05:02:02.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:02:03.084 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:03.085 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:02:03.096 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-13T05:02:01.940211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250372,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754724348,"statfs":{"total":773094113280,"available":772837732352,"internally_reserved":0,"allocated":1466368,"data_stored":1610286,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394849+0000","last_change":"2024-09-13T05:01:59.394849+0000","last_active":"2024-09-13T05:01:59.394849+0000","last_peered":"2024-09-13T05:01:59.394849+0000","last_clean":"2024-09-13T05:01:59.394849+0000","last_became_active":"2024-09-13T05:01:57.178293+0000","last_became_peered":"2024-09-13T05:01:57.178293+0000","last_unstale":"2024-09-13T05:01:59.394849+0000","last_undegraded":"2024-09-13T05:01:59.394849+0000","last_fullsized":"2024-09-13T05:01:59.394849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:03.945702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040222199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394844+0000","last_change":"2024-09-13T05:01:59.394844+0000","last_active":"2024-09-13T05:01:59.394844+0000","last_peered":"2024-09-13T05:01:59.394844+0000","last_clean":"2024-09-13T05:01:59.394844+0000","last_became_active":"2024-09-13T05:01:57.177751+0000","last_became_peered":"2024-09-13T05:01:57.177751+0000","last_unstale":"2024-09-13T05:01:59.394844+0000","last_undegraded":"2024-09-13T05:01:59.394844+0000","last_fullsized":"2024-09-13T05:01:59.394844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:30:48.740698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039273500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565274+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:01:59.565274+0000","last_peered":"2024-09-13T05:01:59.565274+0000","last_clean":"2024-09-13T05:01:59.565274+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:01:59.565274+0000","last_undegraded":"2024-09-13T05:01:59.565274+0000","last_fullsized":"2024-09-13T05:01:59.565274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.564904+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:01:59.564904+0000","last_peered":"2024-09-13T05:01:59.564904+0000","last_clean":"2024-09-13T05:01:59.564904+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:01:59.564904+0000","last_undegraded":"2024-09-13T05:01:59.564904+0000","last_fullsized":"2024-09-13T05:01:59.564904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"16'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195266+0000","last_change":"2024-09-13T05:01:57.175202+0000","last_active":"2024-09-13T05:01:59.195266+0000","last_peered":"2024-09-13T05:01:59.195266+0000","last_clean":"2024-09-13T05:01:59.195266+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:01:59.195266+0000","last_undegraded":"2024-09-13T05:01:59.195266+0000","last_fullsized":"2024-09-13T05:01:59.195266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:29.733421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195889+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:01:59.195889+0000","last_peered":"2024-09-13T05:01:59.195889+0000","last_clean":"2024-09-13T05:01:59.195889+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:01:59.195889+0000","last_undegraded":"2024-09-13T05:01:59.195889+0000","last_fullsized":"2024-09-13T05:01:59.195889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.197848+0000","last_change":"2024-09-13T05:01:59.195603+0000","last_active":"2024-09-13T05:01:59.197848+0000","last_peered":"2024-09-13T05:01:59.197848+0000","last_clean":"2024-09-13T05:01:59.197848+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:01:59.197848+0000","last_undegraded":"2024-09-13T05:01:59.197848+0000","last_fullsized":"2024-09-13T05:01:59.197848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:02.276500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565310+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:01:59.565310+0000","last_peered":"2024-09-13T05:01:59.565310+0000","last_clean":"2024-09-13T05:01:59.565310+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:01:59.565310+0000","last_undegraded":"2024-09-13T05:01:59.565310+0000","last_fullsized":"2024-09-13T05:01:59.565310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195613+0000","last_change":"2024-09-13T05:01:59.195613+0000","last_active":"2024-09-13T05:01:59.195613+0000","last_peered":"2024-09-13T05:01:59.195613+0000","last_clean":"2024-09-13T05:01:59.195613+0000","last_became_active":"2024-09-13T05:01:57.172867+0000","last_became_peered":"2024-09-13T05:01:57.172867+0000","last_unstale":"2024-09-13T05:01:59.195613+0000","last_undegraded":"2024-09-13T05:01:59.195613+0000","last_fullsized":"2024-09-13T05:01:59.195613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:03:56.676979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033540799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":229376,"data_stored":644076,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":644057,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":172032,"data_stored":53708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:02:03.098 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:02:03.378 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:03.379 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:02:03.390 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-13T05:02:01.940211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250372,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754724348,"statfs":{"total":773094113280,"available":772837732352,"internally_reserved":0,"allocated":1466368,"data_stored":1610286,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394849+0000","last_change":"2024-09-13T05:01:59.394849+0000","last_active":"2024-09-13T05:01:59.394849+0000","last_peered":"2024-09-13T05:01:59.394849+0000","last_clean":"2024-09-13T05:01:59.394849+0000","last_became_active":"2024-09-13T05:01:57.178293+0000","last_became_peered":"2024-09-13T05:01:57.178293+0000","last_unstale":"2024-09-13T05:01:59.394849+0000","last_undegraded":"2024-09-13T05:01:59.394849+0000","last_fullsized":"2024-09-13T05:01:59.394849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:03.945702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040222199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394844+0000","last_change":"2024-09-13T05:01:59.394844+0000","last_active":"2024-09-13T05:01:59.394844+0000","last_peered":"2024-09-13T05:01:59.394844+0000","last_clean":"2024-09-13T05:01:59.394844+0000","last_became_active":"2024-09-13T05:01:57.177751+0000","last_became_peered":"2024-09-13T05:01:57.177751+0000","last_unstale":"2024-09-13T05:01:59.394844+0000","last_undegraded":"2024-09-13T05:01:59.394844+0000","last_fullsized":"2024-09-13T05:01:59.394844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:30:48.740698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039273500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565274+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:01:59.565274+0000","last_peered":"2024-09-13T05:01:59.565274+0000","last_clean":"2024-09-13T05:01:59.565274+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:01:59.565274+0000","last_undegraded":"2024-09-13T05:01:59.565274+0000","last_fullsized":"2024-09-13T05:01:59.565274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.564904+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:01:59.564904+0000","last_peered":"2024-09-13T05:01:59.564904+0000","last_clean":"2024-09-13T05:01:59.564904+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:01:59.564904+0000","last_undegraded":"2024-09-13T05:01:59.564904+0000","last_fullsized":"2024-09-13T05:01:59.564904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"16'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195266+0000","last_change":"2024-09-13T05:01:57.175202+0000","last_active":"2024-09-13T05:01:59.195266+0000","last_peered":"2024-09-13T05:01:59.195266+0000","last_clean":"2024-09-13T05:01:59.195266+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:01:59.195266+0000","last_undegraded":"2024-09-13T05:01:59.195266+0000","last_fullsized":"2024-09-13T05:01:59.195266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:29.733421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195889+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:01:59.195889+0000","last_peered":"2024-09-13T05:01:59.195889+0000","last_clean":"2024-09-13T05:01:59.195889+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:01:59.195889+0000","last_undegraded":"2024-09-13T05:01:59.195889+0000","last_fullsized":"2024-09-13T05:01:59.195889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.197848+0000","last_change":"2024-09-13T05:01:59.195603+0000","last_active":"2024-09-13T05:01:59.197848+0000","last_peered":"2024-09-13T05:01:59.197848+0000","last_clean":"2024-09-13T05:01:59.197848+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:01:59.197848+0000","last_undegraded":"2024-09-13T05:01:59.197848+0000","last_fullsized":"2024-09-13T05:01:59.197848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:02.276500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565310+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:01:59.565310+0000","last_peered":"2024-09-13T05:01:59.565310+0000","last_clean":"2024-09-13T05:01:59.565310+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:01:59.565310+0000","last_undegraded":"2024-09-13T05:01:59.565310+0000","last_fullsized":"2024-09-13T05:01:59.565310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195613+0000","last_change":"2024-09-13T05:01:59.195613+0000","last_active":"2024-09-13T05:01:59.195613+0000","last_peered":"2024-09-13T05:01:59.195613+0000","last_clean":"2024-09-13T05:01:59.195613+0000","last_became_active":"2024-09-13T05:01:57.172867+0000","last_became_peered":"2024-09-13T05:01:57.172867+0000","last_unstale":"2024-09-13T05:01:59.195613+0000","last_undegraded":"2024-09-13T05:01:59.195613+0000","last_fullsized":"2024-09-13T05:01:59.195613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:03:56.676979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033540799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":229376,"data_stored":644076,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":644057,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":172032,"data_stored":53708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:02:03.391 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T05:02:03.392 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-13T05:02:03.392 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-13T05:02:03.392 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-13T05:02:03.712 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:03.712 INFO:teuthology.orchestra.run.smithi053.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-13T05:02:03.724 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-13T05:02:03.725 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-13T05:02:03.737 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-13T05:02:03.737 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache max dirty': 0}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '48a7de7e6e1701a88fd571c7e34908ac4b8ed412'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': '7a853b1706d076d58e0da7a5d3b9b9eb36276880'}} 2024-09-13T05:02:03.737 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-13T05:02:03.738 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-13T05:02:04.041 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-13T05:02:04.041 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T05:02:04.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:02:04.324 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:04.324 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:02:04.336 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-13T05:02:03.940672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250372,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754724348,"statfs":{"total":773094113280,"available":772837732352,"internally_reserved":0,"allocated":1466368,"data_stored":1610286,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000460"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394849+0000","last_change":"2024-09-13T05:01:59.394849+0000","last_active":"2024-09-13T05:01:59.394849+0000","last_peered":"2024-09-13T05:01:59.394849+0000","last_clean":"2024-09-13T05:01:59.394849+0000","last_became_active":"2024-09-13T05:01:57.178293+0000","last_became_peered":"2024-09-13T05:01:57.178293+0000","last_unstale":"2024-09-13T05:01:59.394849+0000","last_undegraded":"2024-09-13T05:01:59.394849+0000","last_fullsized":"2024-09-13T05:01:59.394849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:03.945702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040222199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394844+0000","last_change":"2024-09-13T05:01:59.394844+0000","last_active":"2024-09-13T05:01:59.394844+0000","last_peered":"2024-09-13T05:01:59.394844+0000","last_clean":"2024-09-13T05:01:59.394844+0000","last_became_active":"2024-09-13T05:01:57.177751+0000","last_became_peered":"2024-09-13T05:01:57.177751+0000","last_unstale":"2024-09-13T05:01:59.394844+0000","last_undegraded":"2024-09-13T05:01:59.394844+0000","last_fullsized":"2024-09-13T05:01:59.394844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:30:48.740698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039273500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565274+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:01:59.565274+0000","last_peered":"2024-09-13T05:01:59.565274+0000","last_clean":"2024-09-13T05:01:59.565274+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:01:59.565274+0000","last_undegraded":"2024-09-13T05:01:59.565274+0000","last_fullsized":"2024-09-13T05:01:59.565274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.564904+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:01:59.564904+0000","last_peered":"2024-09-13T05:01:59.564904+0000","last_clean":"2024-09-13T05:01:59.564904+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:01:59.564904+0000","last_undegraded":"2024-09-13T05:01:59.564904+0000","last_fullsized":"2024-09-13T05:01:59.564904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"16'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195266+0000","last_change":"2024-09-13T05:01:57.175202+0000","last_active":"2024-09-13T05:01:59.195266+0000","last_peered":"2024-09-13T05:01:59.195266+0000","last_clean":"2024-09-13T05:01:59.195266+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:01:59.195266+0000","last_undegraded":"2024-09-13T05:01:59.195266+0000","last_fullsized":"2024-09-13T05:01:59.195266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:29.733421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195889+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:01:59.195889+0000","last_peered":"2024-09-13T05:01:59.195889+0000","last_clean":"2024-09-13T05:01:59.195889+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:01:59.195889+0000","last_undegraded":"2024-09-13T05:01:59.195889+0000","last_fullsized":"2024-09-13T05:01:59.195889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.197848+0000","last_change":"2024-09-13T05:01:59.195603+0000","last_active":"2024-09-13T05:01:59.197848+0000","last_peered":"2024-09-13T05:01:59.197848+0000","last_clean":"2024-09-13T05:01:59.197848+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:01:59.197848+0000","last_undegraded":"2024-09-13T05:01:59.197848+0000","last_fullsized":"2024-09-13T05:01:59.197848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:02.276500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565310+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:01:59.565310+0000","last_peered":"2024-09-13T05:01:59.565310+0000","last_clean":"2024-09-13T05:01:59.565310+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:01:59.565310+0000","last_undegraded":"2024-09-13T05:01:59.565310+0000","last_fullsized":"2024-09-13T05:01:59.565310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195613+0000","last_change":"2024-09-13T05:01:59.195613+0000","last_active":"2024-09-13T05:01:59.195613+0000","last_peered":"2024-09-13T05:01:59.195613+0000","last_clean":"2024-09-13T05:01:59.195613+0000","last_became_active":"2024-09-13T05:01:57.172867+0000","last_became_peered":"2024-09-13T05:01:57.172867+0000","last_unstale":"2024-09-13T05:01:59.195613+0000","last_undegraded":"2024-09-13T05:01:59.195613+0000","last_fullsized":"2024-09-13T05:01:59.195613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:03:56.676979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033540799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":229376,"data_stored":644076,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":644057,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":172032,"data_stored":53708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:02:04.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:02:04.617 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:04.618 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:02:04.629 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-13T05:02:03.940672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":250372,"kb_used_data":1432,"kb_used_omap":12,"kb_used_meta":248819,"kb_avail":754724348,"statfs":{"total":773094113280,"available":772837732352,"internally_reserved":0,"allocated":1466368,"data_stored":1610286,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":254791225},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000460"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394849+0000","last_change":"2024-09-13T05:01:59.394849+0000","last_active":"2024-09-13T05:01:59.394849+0000","last_peered":"2024-09-13T05:01:59.394849+0000","last_clean":"2024-09-13T05:01:59.394849+0000","last_became_active":"2024-09-13T05:01:57.178293+0000","last_became_peered":"2024-09-13T05:01:57.178293+0000","last_unstale":"2024-09-13T05:01:59.394849+0000","last_undegraded":"2024-09-13T05:01:59.394849+0000","last_fullsized":"2024-09-13T05:01:59.394849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:03.945702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040222199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.394844+0000","last_change":"2024-09-13T05:01:59.394844+0000","last_active":"2024-09-13T05:01:59.394844+0000","last_peered":"2024-09-13T05:01:59.394844+0000","last_clean":"2024-09-13T05:01:59.394844+0000","last_became_active":"2024-09-13T05:01:57.177751+0000","last_became_peered":"2024-09-13T05:01:57.177751+0000","last_unstale":"2024-09-13T05:01:59.394844+0000","last_undegraded":"2024-09-13T05:01:59.394844+0000","last_fullsized":"2024-09-13T05:01:59.394844+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:30:48.740698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039273500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565274+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:01:59.565274+0000","last_peered":"2024-09-13T05:01:59.565274+0000","last_clean":"2024-09-13T05:01:59.565274+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:01:59.565274+0000","last_undegraded":"2024-09-13T05:01:59.565274+0000","last_fullsized":"2024-09-13T05:01:59.565274+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.564904+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:01:59.564904+0000","last_peered":"2024-09-13T05:01:59.564904+0000","last_clean":"2024-09-13T05:01:59.564904+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:01:59.564904+0000","last_undegraded":"2024-09-13T05:01:59.564904+0000","last_fullsized":"2024-09-13T05:01:59.564904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"16'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195266+0000","last_change":"2024-09-13T05:01:57.175202+0000","last_active":"2024-09-13T05:01:59.195266+0000","last_peered":"2024-09-13T05:01:59.195266+0000","last_clean":"2024-09-13T05:01:59.195266+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:01:59.195266+0000","last_undegraded":"2024-09-13T05:01:59.195266+0000","last_fullsized":"2024-09-13T05:01:59.195266+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:29.733421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195889+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:01:59.195889+0000","last_peered":"2024-09-13T05:01:59.195889+0000","last_clean":"2024-09-13T05:01:59.195889+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:01:59.195889+0000","last_undegraded":"2024-09-13T05:01:59.195889+0000","last_fullsized":"2024-09-13T05:01:59.195889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.197848+0000","last_change":"2024-09-13T05:01:59.195603+0000","last_active":"2024-09-13T05:01:59.197848+0000","last_peered":"2024-09-13T05:01:59.197848+0000","last_clean":"2024-09-13T05:01:59.197848+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:01:59.197848+0000","last_undegraded":"2024-09-13T05:01:59.197848+0000","last_fullsized":"2024-09-13T05:01:59.197848+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:02.276500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.565310+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:01:59.565310+0000","last_peered":"2024-09-13T05:01:59.565310+0000","last_clean":"2024-09-13T05:01:59.565310+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:01:59.565310+0000","last_undegraded":"2024-09-13T05:01:59.565310+0000","last_fullsized":"2024-09-13T05:01:59.565310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-13T05:01:59.195613+0000","last_change":"2024-09-13T05:01:59.195613+0000","last_active":"2024-09-13T05:01:59.195613+0000","last_peered":"2024-09-13T05:01:59.195613+0000","last_clean":"2024-09-13T05:01:59.195613+0000","last_became_active":"2024-09-13T05:01:57.172867+0000","last_became_peered":"2024-09-13T05:01:57.172867+0000","last_unstale":"2024-09-13T05:01:59.195613+0000","last_undegraded":"2024-09-13T05:01:59.195613+0000","last_fullsized":"2024-09-13T05:01:59.195613+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:03:56.676979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033540799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31348,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":229376,"data_stored":644076,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31336,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":225280,"data_stored":644057,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31280,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340560,"statfs":{"total":96636764160,"available":96604733440,"internally_reserved":0,"allocated":172032,"data_stored":53708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:02:04.630 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T05:02:04.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:epoch 18 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:fsid 01e1476b-1467-4e0e-b01c-0f8cdf8276fe 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:created 2024-09-13T05:01:45.075461+0000 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:modified 2024-09-13T05:02:04.862972+0000 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:crush_version 4 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:full_ratio 0.95 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:backfillfull_ratio 0.9 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:nearfull_ratio 0.85 2024-09-13T05:02:04.917 INFO:teuthology.orchestra.run.smithi053.stdout:require_min_compat_client luminous 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:min_compat_client jewel 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:require_osd_release reef 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:stretch_mode_enabled false 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:max_osd 8 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/1769628429,v1:172.21.15.53:6817/1769628429] [v2:172.21.15.53:6818/1769628429,v1:172.21.15.53:6819/1769628429] exists,up baf16a5c-4204-424b-8d8e-a5b1486aa052 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/827277153,v1:172.21.15.53:6801/827277153] [v2:172.21.15.53:6802/827277153,v1:172.21.15.53:6803/827277153] exists,up f3df9a0f-3a9e-4665-9233-718bc0fe1223 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/1334123504,v1:172.21.15.53:6809/1334123504] [v2:172.21.15.53:6810/1334123504,v1:172.21.15.53:6811/1334123504] exists,up 1304a876-75c5-454b-a60b-8c9823c4aca8 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/4134389313,v1:172.21.15.53:6825/4134389313] [v2:172.21.15.53:6826/4134389313,v1:172.21.15.53:6827/4134389313] exists,up 83668012-8ddb-4fe5-bb31-6e78d2723f12 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/3681797816,v1:172.21.15.110:6809/3681797816] [v2:172.21.15.110:6810/3681797816,v1:172.21.15.110:6811/3681797816] exists,up f462232f-5144-4652-9139-9039d53e191a 2024-09-13T05:02:04.918 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/81482652,v1:172.21.15.110:6801/81482652] [v2:172.21.15.110:6802/81482652,v1:172.21.15.110:6803/81482652] exists,up 1d2d8607-f1d9-45e3-8486-3f5bb23b516b 2024-09-13T05:02:04.919 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6814/576600432,v1:172.21.15.110:6817/576600432] [v2:172.21.15.110:6818/576600432,v1:172.21.15.110:6819/576600432] exists,up 7fe0fb7d-0a40-4666-82a1-4e18f53b445c 2024-09-13T05:02:04.919 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/896779674,v1:172.21.15.110:6825/896779674] [v2:172.21.15.110:6826/896779674,v1:172.21.15.110:6827/896779674] exists,up 811503be-abe3-4d64-a094-09ff675f3666 2024-09-13T05:02:04.932 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6816/1769628429,v1:172.21.15.53:6817/1769628429] [v2:172.21.15.53:6818/1769628429,v1:172.21.15.53:6819/1769628429] exists,up baf16a5c-4204-424b-8d8e-a5b1486aa052', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6800/827277153,v1:172.21.15.53:6801/827277153] [v2:172.21.15.53:6802/827277153,v1:172.21.15.53:6803/827277153] exists,up f3df9a0f-3a9e-4665-9233-718bc0fe1223', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6808/1334123504,v1:172.21.15.53:6809/1334123504] [v2:172.21.15.53:6810/1334123504,v1:172.21.15.53:6811/1334123504] exists,up 1304a876-75c5-454b-a60b-8c9823c4aca8', 'osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.53:6824/4134389313,v1:172.21.15.53:6825/4134389313] [v2:172.21.15.53:6826/4134389313,v1:172.21.15.53:6827/4134389313] exists,up 83668012-8ddb-4fe5-bb31-6e78d2723f12', 'osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/3681797816,v1:172.21.15.110:6809/3681797816] [v2:172.21.15.110:6810/3681797816,v1:172.21.15.110:6811/3681797816] exists,up f462232f-5144-4652-9139-9039d53e191a', 'osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/81482652,v1:172.21.15.110:6801/81482652] [v2:172.21.15.110:6802/81482652,v1:172.21.15.110:6803/81482652] exists,up 1d2d8607-f1d9-45e3-8486-3f5bb23b516b', 'osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6814/576600432,v1:172.21.15.110:6817/576600432] [v2:172.21.15.110:6818/576600432,v1:172.21.15.110:6819/576600432] exists,up 7fe0fb7d-0a40-4666-82a1-4e18f53b445c', 'osd.7 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/896779674,v1:172.21.15.110:6825/896779674] [v2:172.21.15.110:6826/896779674,v1:172.21.15.110:6827/896779674] exists,up 811503be-abe3-4d64-a094-09ff675f3666'] 2024-09-13T05:02:04.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-13T05:02:05.034 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:05.034 INFO:teuthology.orchestra.run.smithi053.stdout: "name": "mon.a", 2024-09-13T05:02:05.034 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster": "ceph", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "admin_socket_mode": "", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_client_required": "cephx, none", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_cluster_required": "cephx", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_debug": "true", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_service_required": "cephx", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "auth_supported": "cephx", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio": "true", 2024-09-13T05:02:05.035 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_poll_ms": "250", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_aio_reap_max": "16", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_async_discard": "false", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_block_size": "4096", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio": "false", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_debug_inflight_ios": "false", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_enable_discard": "false", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_flock_retry": "3", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-13T05:02:05.036 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_inject_crash": "0", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring": "false", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring_hipri": "false", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bdev_type": "", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_alloc_size": "1048576", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_allocator": "hybrid", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_buffered_io": "true", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_for_zeros": "false", 2024-09-13T05:02:05.037 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_compact_log_sync": "false", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_max_log_runway": "4194304", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_max_prefetch": "1048576", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_min_flush_size": "524288", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_min_log_runway": "1048576", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_replay_recovery": "false", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-13T05:02:05.038 INFO:teuthology.orchestra.run.smithi053.stdout: "bluefs_sync_write": "false", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_allocation_from_file": "true", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_allocator": "hybrid", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_create": "true", 2024-09-13T05:02:05.039 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_create": "false", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_path": "", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_db_size": "0", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_path": "", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_preallocate_file": "false", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_size": "107374182400", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_create": "false", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_path": "", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_block_wal_size": "100663296", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs": "true", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-13T05:02:05.040 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_autotune": "true", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size": "0", 2024-09-13T05:02:05.041 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cache_type": "2q", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_clone_cow": "true", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-13T05:02:05.042 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_mode": "none", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_csum_type": "crc32c", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_freelist": "false", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-13T05:02:05.043 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_small_allocations": "0", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_default_buffered_read": "true", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_default_buffered_write": "false", 2024-09-13T05:02:05.044 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fail_eio": "false", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-13T05:02:05.045 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mount": "false", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_umount": "false", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-13T05:02:05.046 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_ignore_data_csum": "false", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_log_op_age": "5.000000", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_alloc_size": "0", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size": "0", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_max_deferred_txc": "32", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size": "0", 2024-09-13T05:02:05.047 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_nid_prealloc": "1024", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_retry_disk_reads": "3", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_cf": "true", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-13T05:02:05.048 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_spdk_mem": "512", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_tracing": "false", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-13T05:02:05.049 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "bluestore_zero_block_detection": "false", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-13T05:02:05.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_cluster_require_signatures": "false", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_cluster_require_version": "2", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_require_signatures": "false", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_require_version": "2", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_service_require_signatures": "false", 2024-09-13T05:02:05.051 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_service_require_version": "2", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "cephx_sign_messages": "true", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "chdir": "", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_acl_type": "", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_asio_thread_count": "2", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_cache_mid": "0.750000", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_cache_size": "16384", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_caps_release_delay": "5", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_check_pool_perm": "true", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_force_sync_read": "false", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_getattr_caps": "false", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_inject_features": "", 2024-09-13T05:02:05.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_debug_inject_tick_delay": "0", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_die_on_failed_remount": "false", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_dirsize_rbytes": "true", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_force_lazyio": "false", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_fs": "", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-13T05:02:05.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_inject_release_failure": "false", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_max_inline_size": "4096", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mds_namespace": "", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_metadata": "", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_gid": "-1", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_timeout": "300", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mount_uid": "-1", 2024-09-13T05:02:05.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_mountpoint": "/", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_notify_timeout": "10", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc": "true", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_dirty": "104857600", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_max_objects": "1000", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_size": "209715200", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_oc_target_dirty": "8388608", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_permissions": "true", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_quota": "true", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_quota_df": "true", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_max_bytes": "0", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_max_periods": "4", 2024-09-13T05:02:05.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_readahead_min": "131072", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_reconnect_stale": "false", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_shutdown_timeout": "30", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_snapdir": ".snap", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_tick_interval": "1", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_trace": "", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_try_dentry_invalidate": "false", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_use_faked_inos": "false", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_use_random_mds": "false", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog": "false", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_graylog_port": "12201", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_monitors": "default=true", 2024-09-13T05:02:05.056 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog": "false", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "clog_to_syslog_level": "info", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "-", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_network": "", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_network_interface": "", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_isal": "false", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_level": "5", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zlib_winsize": "-15", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "compressor_zstd_level": "1", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_alien_op_num_threads": "6", 2024-09-13T05:02:05.057 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "crimson_seastar_smp": "1", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location": "", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location_hook": "", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_location_hook_timeout": "10", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "daemonize": "false", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "daos_pool": "tank", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_alienstore": "0/5", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_allow_any_pool_priority": "false", 2024-09-13T05:02:05.058 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asok": "1/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asok_assert_abort": "false", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_asserts_on_shutdown": "true", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_auth": "20/20", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bdev": "1/3", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bluefs": "1/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_bluestore": "1/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_buffer": "0/1", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_ceph_exporter": "1/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cephfs_mirror": "0/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cephsqlite": "0/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_client": "0/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_compressor": "1/5", 2024-09-13T05:02:05.059 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_context": "0/1", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crush": "1/1", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_crypto": "1/5", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_cyanstore": "0/5", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_deliberately_leak_memory": "false", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_disable_randomized_ping": "false", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_dpdk": "1/5", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_eventtrace": "1/5", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_filer": "0/1", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_filestore": "1/3", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_finisher": "1/1", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_fuse": "1/5", 2024-09-13T05:02:05.060 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_heartbeat_testing_span": "0", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_heartbeatmap": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_javaclient": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_journal": "1/3", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_journaler": "0/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_kstore": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_leveldb": "4/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_lockdep": "0/1", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mclock": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_balancer": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_locker": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_log": "1/5", 2024-09-13T05:02:05.061 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_log_expire": "1/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mds_migrator": "1/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_memstore": "1/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mgr": "2/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mgrc": "1/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_mon": "20/20", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_monc": "0/10", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_ms": "1/1", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_none": "0/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objclass": "0/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objectcacher": "0/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_objecter": "0/1", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_optracker": "0/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_osd": "1/5", 2024-09-13T05:02:05.062 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_paxos": "20/20", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_perfcounter": "1/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_prioritycache": "1/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rados": "0/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd": "0/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_mirror": "0/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_pwl": "0/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rbd_replay": "0/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_refs": "0/0", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_reserver": "1/1", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw": "1/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_access": "1/5", 2024-09-13T05:02:05.063 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_datacache": "1/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_dbstore": "1/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_flight": "1/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rgw_sync": "1/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_rocksdb": "4/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_backref": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_cache": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_cleaner": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_device": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_epm": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_journal": "0/5", 2024-09-13T05:02:05.064 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_lba": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_odata": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_omap": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_onode": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_t": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_seastore_tm": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_striper": "0/1", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_test": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_throttle": "1/1", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_timer": "0/1", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "debug_tp": "0/5", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "device_failure_prediction_mode": "none", 2024-09-13T05:02:05.065 INFO:teuthology.orchestra.run.smithi053.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_graylog": "false", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_journald": "false", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_stderr": "true", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "err_to_syslog": "false", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event_tracing": "false", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_addr": "0.0.0.0", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_http_port": "9926", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_prio_limit": "5", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_sort_metrics": "true", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exporter_stats_period": "5", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "fake_statfs_for_testing": "0", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "fatal_signal_handlers": "true", 2024-09-13T05:02:05.066 INFO:teuthology.orchestra.run.smithi053.stdout: "filer_max_purge_ops": "10", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filer_max_truncate_ops": "128", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_apply_finisher_threads": "1", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_blackhole": "false", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_btrfs_clone_range": "true", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_btrfs_snap": "true", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_caller_concurrency": "10", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_collect_device_partition_information": "true", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_commit_timeout": "600.000000", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_inject_read_err": "false", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_omap_check": "false", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_debug_verify_split": "false", 2024-09-13T05:02:05.067 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_dump_file": "", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fadvise": "true", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fail_eio": "true", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fd_cache_shards": "16", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fd_cache_size": "128", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fiemap": "false", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fiemap_threshold": "4096", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_inject_stall": "0", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_parallel": "false", 2024-09-13T05:02:05.068 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_trailing": "false", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_journal_writeahead": "false", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_kill_at": "0", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs": "0", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size": "0", 2024-09-13T05:02:05.069 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_merge_threshold": "-10", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_odsync_write": "false", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_backend": "rocksdb", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_backend_path": "", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_thread_timeout": "60", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_op_threads": "2", 2024-09-13T05:02:05.070 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_punch_hole": "false", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-13T05:02:05.071 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_queue_max_ops": "50", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_seek_data_hole": "false", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_sloppy_crc": "false", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_splice": "false", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_split_multiple": "2", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_split_rand_factor": "20", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_update_to": "1000", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-13T05:02:05.072 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_enable": "true", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_xfs_extsize": "false", 2024-09-13T05:02:05.073 INFO:teuthology.orchestra.run.smithi053.stdout: "filestore_zfs_snap": "false", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fio_dir": "/tmp/fio", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "01e1476b-1467-4e0e-b01c-0f8cdf8276fe", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_allow_other": "true", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_atomic_o_trunc": "true", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_big_writes": "true", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_debug": "false", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_default_permissions": "false", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_disable_pagecache": "false", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_max_write": "0", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_multithreaded": "true", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_require_active_mds": "true", 2024-09-13T05:02:05.074 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_set_user_groups": "true", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_move": "true", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_read": "true", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_splice_write": "true", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "fuse_use_invalidate_cb": "true", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "gss_target_name": "ceph", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_file": "", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_inject_failure": "0", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_interval": "5", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "host": "smithi053", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-13T05:02:05.075 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-13T05:02:05.076 INFO:teuthology.orchestra.run.smithi053.stdout: "inject_early_sigterm": "false", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "jaeger_agent_port": "6799", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "jaeger_tracing_enable": "false", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_aio": "true", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_align_min_size": "65536", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_block_align": "true", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_block_size": "4096", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_dio": "true", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_discard": "false", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_force_aio": "false", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_ignore_corruption": "false", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_max_write_bytes": "10485760", 2024-09-13T05:02:05.077 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_max_write_entries": "100", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_replay_from": "0", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_write_header_frequency": "0", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journal_zero_on_create": "false", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_prefetch_periods": "10", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_prezero_periods": "5", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "journaler_write_head_interval": "15", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "key": "", 2024-09-13T05:02:05.078 INFO:teuthology.orchestra.run.smithi053.stdout: "keyfile": "", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_backend": "rocksdb", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_default_stripe_size": "65536", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_fsck_on_mount": "false", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_max_bytes": "67108864", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_max_ops": "512", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_nid_prealloc": "1024", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_onode_map_size": "1024", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_sync_submit_transaction": "false", 2024-09-13T05:02:05.079 INFO:teuthology.orchestra.run.smithi053.stdout: "kstore_sync_transaction": "false", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_block_size": "65536", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_bloom_size": "0", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_cache_size": "536870912", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_compact_on_mount": "false", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_compression": "false", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_log": "", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_max_open_files": "0", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_paranoid": "false", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "librados_thread_count": "2", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "lockdep": "false", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "lockdep_force_backtrace": "false", 2024-09-13T05:02:05.080 INFO:teuthology.orchestra.run.smithi053.stdout: "log_coarse_timestamps": "true", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_flush_on_exit": "false", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_graylog_host": "127.0.0.1", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_graylog_port": "12201", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_max_new": "1000", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_max_recent": "10000", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_stderr_prefix": "", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_stop_at_utilization": "0.970000", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_file": "true", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_graylog": "false", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_journald": "false", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_stderr": "false", 2024-09-13T05:02:05.081 INFO:teuthology.orchestra.run.smithi053.stdout: "log_to_syslog": "false", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "max_rotating_auth_attempts": "10", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_action_on_write_error": "1", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_alternate_name_max": "8192", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_asio_thread_count": "2", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_export_pin": "true", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_dirs": "true", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_interval": "5", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_interval": "10", 2024-09-13T05:02:05.082 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_max": "-1", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_max_until": "-1", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_merge_size": "50", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_midchunk": "0.300000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_min_start": "0.200000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_minchunk": "0.001000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_mode": "0", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_need_max": "1.200000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_need_min": "0.800000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_bits": "3", 2024-09-13T05:02:05.083 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_size": "10000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_target_decay": "10.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_grace": "15.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_interval": "4.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_mid": "0.700000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_release_free_interval": "10", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_reservation": "0.050000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-13T05:02:05.084 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_interval": "1", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cache_trim_threshold": "262144", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_prealloc_inos": "1000", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_connect_bootstrapping": "false", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_damage_table_max_entries": "10000", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_auth_pins": "false", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_frag": "false", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_scatterstat": "false", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_debug_subtrees": "false", 2024-09-13T05:02:05.085 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_decay_halflife": "5.000000", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_default_dir_hash": "2", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_defer_session_stale": "true", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_deny_all_reconnect": "false", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_keys_per_op": "16384", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_max_commit_size": "10", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_max_entries": "0", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dir_prefetch": "true", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_on_map": "false", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_early_reply": "true", 2024-09-13T05:02:05.086 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_enable_op_tracker": "true", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_enforce_unique_name": "true", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_random": "true", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_health_summarize_threshold": "10", 2024-09-13T05:02:05.087 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_health_dummy": "false", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_migrator_session_race": "false", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_join_fs": "", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_journal_format": "1", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_create_at": "0", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_export_at": "0", 2024-09-13T05:02:05.088 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_import_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_expire_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_journal_replay_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_link_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_mdstable_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_openc_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_kill_rename_at": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_events_per_segment": "1024", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_max_events": "-1", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_max_segments": "128", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_pause": "false", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_segment_size": "0", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-13T05:02:05.089 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_log_warn_factor": "2.000000", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_caps_per_client": "1048576", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_completed_flushes": "100000", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_completed_requests": "100000", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_export_size": "20971520", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_file_recover": "32", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_files": "64", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_ops": "8192", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_max_snaps_per_dir": "100", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_metrics_update_interval": "2", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_min_caps_per_client": "100", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_min_caps_working_set": "10000", 2024-09-13T05:02:05.090 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_numa_node": "-1", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_complaint_time": "30.000000", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_duration": "600", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_size": "20", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_slow_op_size": "20", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_op_log_threshold": "5", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_ping_grace": "15", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_ping_interval": "5", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-13T05:02:05.091 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_caps": "30000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_recall_warning_threshold": "262144", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_replay_interval": "1.000000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_root_ino_gid": "0", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_root_ino_uid": "0", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-13T05:02:05.092 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_shutdown_check": "0", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_skip_ino": "0", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_max_uid": "4294967294", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_min_uid": "0", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_snap_rstat": "false", 2024-09-13T05:02:05.093 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_standby_replay_damaged": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_symlink_recovery": "true", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_thrash_exports": "0", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_thrash_fragments": "0", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_tick_interval": "5.000000", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_valgrind_exit": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_verify_backtrace": "1", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_verify_scatter": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mds_wipe_sessions": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "mempool_debug": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_device_bytes": "1073741824", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_page_set": "false", 2024-09-13T05:02:05.094 INFO:teuthology.orchestra.run.smithi053.stdout: "memstore_page_size": "65536", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_bytes": "134217728", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_messages": "512", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_disabled_modules": "", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_max_pg_num_change": "128", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mds_bytes": "134217728", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mds_messages": "128", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mon_bytes": "134217728", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_mon_messages": "128", 2024-09-13T05:02:05.095 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_osd_bytes": "536870912", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_osd_messages": "8192", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_pool": "true", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_standby_modules": "true", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_stats_period": "5", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_stats_threshold": "5", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_tick_period": "2", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_allow_pool_delete": "true", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_allow_pool_size_one": "true", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_auth_validate_all_caps": "true", 2024-09-13T05:02:05.096 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_bytes": "104857600", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_directed_command_retry": "5", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_hunt_parallel": "3", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_log_interval": "1.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_ping_interval": "10.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-13T05:02:05.097 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_file_level": "debug", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_file": "true", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_journald": "false", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_bootstrap": "false", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_start": "false", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_compact_on_trim": "true", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-13T05:02:05.098 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_cpu_threads": "4", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_crush_min_required_version": "hammer", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_daemon_bytes": "419430400", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_avail_crit": "5", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_avail_warn": "5", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_data_size_warn": "16106127360", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_json": "false", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-13T05:02:05.099 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_dump_transactions": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_extra_checks": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_quincy": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_no_require_reef": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_down_mkfs_grace": "120", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_election_default_strategy": "1", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_election_timeout": "5.000000", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-13T05:02:05.100 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ping_divisor": "2", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_enable_op_tracker": "true", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_fake_pool_delete": "false", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_force_quorum_join": "false", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_globalid_prealloc": "10000", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_detail_to_clog": "false", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_log_update_period": "5", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_max_detail": "50", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog": "true", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog_interval": "600", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_host": "172.21.15.53,[v2:172.21.15.53:3301,v1:172.21.15.53:6790],172.21.15.110", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_host_override": "", 2024-09-13T05:02:05.101 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_initial_members": "", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease": "5.000000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_full_interval": "50", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_max": "10000", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_log_max_summary": "50", 2024-09-13T05:02:05.102 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_log_epochs": "500", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_osd": "10000", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_pg_per_osd": "10000", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_pool_pg_num": "65536", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_force_trim_to": "0", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mds_skip_sanity": "false", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_memory_autotune": "true", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_memory_target": "2147483648", 2024-09-13T05:02:05.103 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_beacon_grace": "30", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_digest_period": "5", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_inactive_grace": "60", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_min_osdmap_epochs": "500", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_complaint_time": "30", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_duration": "600", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_size": "20", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_slow_op_size": "20", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-13T05:02:05.104 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_op_log_threshold": "5", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_in": "false", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_cache_size": "500", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_down_out_interval": "600", 2024-09-13T05:02:05.105 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_force_trim_to": "0", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_down_reporters": "2", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-13T05:02:05.106 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_report_timeout": "900", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_stuck_threshold": "60", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-13T05:02:05.107 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_probe_timeout": "2.000000", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_max_change": "0.050000", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_max_osds": "4", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-13T05:02:05.108 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_interval": "86400", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_max_keys": "100", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_scrub_timeout": "300", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_session_timeout": "300", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_smart_report_timeout": "5", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stat_smooth_intervals": "6", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_pool_min_size": "2", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_pool_size": "4", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_debug": "false", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-13T05:02:05.109 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_provider_kill_at": "0", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_requester_kill_at": "0", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_sync_timeout": "60.000000", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_target_pg_per_osd": "100", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_tick_interval": "5", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_timecheck_interval": "300.000000", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_use_min_delay_socket": "false", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_older_version_delay": "604800", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-13T05:02:05.110 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_misplaced": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_older_version": "true", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-13T05:02:05.111 INFO:teuthology.orchestra.run.smithi053.stdout: "monmap": "", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "motr_tracing_enabled": "false", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_op_threads": "3", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_cm": "false", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_device_name": "", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_dscp": "96", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-13T05:02:05.112 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_local_gid": "", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_port_num": "1", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_sl": "3", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_support_srq": "true", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_rdma_type": "ib", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_async_reap_threshold": "5", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_before_connect": "false", 2024-09-13T05:02:05.113 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_ipv4": "true", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_ipv6": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_msgr1": "true", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_msgr2": "true", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_port_max": "7568", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_port_min": "6800", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_retry_count": "3", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_bind_retry_delay": "5", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_client": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mds": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mgr": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_mon": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_blackhole_osd": "false", 2024-09-13T05:02:05.114 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_client_mode": "crc secure", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_cluster_mode": "crc secure", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_cluster_type": "", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_compress_secure": "false", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_connection_idle_timeout": "900", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_connection_ready_timeout": "10", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_crc_data": "true", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_crc_header": "true", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_bad_msg": "false", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_bug": "true", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_old_message": "true", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_skipped_message": "false", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-13T05:02:05.115 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_coremask": "0xF", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_enable_tso": "true", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hugepages": "", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_lro": "true", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_memory_channel": "4", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_pmd": "", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_port_id": "0", 2024-09-13T05:02:05.116 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_dump_on_send": "false", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_initial_backoff": "0.200000", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_max": "1.000000", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_delay_type": "", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_network_congestion": "0", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_inject_socket_failures": "5000", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_learn_addr_from_peer": "true", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_max_accept_failures": "4", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_max_backoff": "15.000000", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_client_mode": "secure crc", 2024-09-13T05:02:05.117 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_mon_service_mode": "secure crc", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_osd_compress_min_size": "1024", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_osd_compress_mode": "none", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_pq_min_cost": "65536", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_public_type": "", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_service_mode": "crc secure", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_listen_backlog": "512", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_nodelay": "true", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-13T05:02:05.118 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_tcp_rcvbuf": "0", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "ms_type": "async+posix", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "no_config_file": "false", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_completion_locks_per_session": "32", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inflight_ops": "1024", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_tick_interval": "5.000000", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objecter_timeout": "10.000000", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "objectstore_blackhole": "false", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "openssl_engine_opts": "", 2024-09-13T05:02:05.119 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_delay_time": "5.000000", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_hist_halflife": "1000", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_max_low_ops": "2", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_max_ops": "4", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_agent_slop": "0.020000", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_asio_thread_count": "2", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_async_recovery_min_cost": "100", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-13T05:02:05.120 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_scan_max": "512", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backfill_scan_min": "64", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_degraded": "false", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_peering": "false", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_backoff_on_unfound": "true", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_beacon_report_interval": "300", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_duration": "30", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_max_block_size": "67108864", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_blkin_trace_all": "false", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-13T05:02:05.121 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_check_for_log_corruption": "false", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-13T05:02:05.122 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_class_update_on_start": "true", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_message_cap": "256", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_message_size_cap": "524288000", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_op_priority": "63", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_client_watch_timeout": "30", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_command_max_records": "256", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_compact_on_start": "false", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_update_on_start": "true", 2024-09-13T05:02:05.123 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_crush_update_weight_set": "true", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_feed_pullee": "-1", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_misdirected_ops": "false", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_no_acting_change": "false", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_no_purge_strays": "false", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_op_order": "false", 2024-09-13T05:02:05.124 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_shutdown": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_snaps": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-13T05:02:05.125 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_keys": "1024", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_stride": "524288", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_default_notify_timeout": "30", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep": "0.000000", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_discard_disconnected_ops": "true", 2024-09-13T05:02:05.126 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_enable_op_tracker": "true", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_info": "true", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown": "true", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_function_tracing": "false", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_grace": "20", 2024-09-13T05:02:05.127 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_interval": "6", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_peers": "10", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_min_size": "2000", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_stale": "600", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_max_size": "100000", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_min_size": "1000", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-13T05:02:05.128 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_journal_size": "5120", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_kill_backfill_at": "0", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_cache_size": "50", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_dedup": "true", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_message_max": "40", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_map_share_max_epochs": "40", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_attr_name_len": "100", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_attr_size": "0", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_backfills": "1", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_markdown_count": "5", 2024-09-13T05:02:05.129 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_markdown_period": "600", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_name_len": "2048", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_namespace_len": "256", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_object_size": "134217728", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_blocked_by": "16", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_log_entries": "10000", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_pgls": "1024", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_push_cost": "8388608", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_push_objects": "10", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_scrubs": "3", 2024-09-13T05:02:05.130 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_trimming_pgs": "2", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_write_op_reply_len": "64", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_max_write_size": "90", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-13T05:02:05.131 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_profile": "balanced", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-13T05:02:05.132 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_base": "805306368", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_cache_min": "134217728", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target": "4294967296", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target_autotune": "false", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_min_pg_log_entries": "250", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_min_recovery_priority": "0", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-13T05:02:05.133 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_report_interval": "5", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_num_cache_shards": "32", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_num_op_tracker_shard": "32", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_auto_affinity": "true", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_node": "-1", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_numa_prefer_iface": "true", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objecter_finishers": "1", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore": "bluestore", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore_fuse": "false", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_objectstore_tracing": "false", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_complaint_time": "30.000000", 2024-09-13T05:02:05.134 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_duration": "600", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_size": "20", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_slow_op_size": "20", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_log_threshold": "5", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards": "0", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards_hdd": "5", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_shards_ssd": "8", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_pq_min_cost": "65536", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-13T05:02:05.135 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_queue_cut_off": "high", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_op_thread_timeout": "15", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_open_classes_on_start": "true", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_os_flags": "0", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_peering_op_priority": "255", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_delete_cost": "1048576", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_delete_priority": "5", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_trim_max": "10000", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_log_trim_min": "100", 2024-09-13T05:02:05.136 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_crimson": "false", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-13T05:02:05.137 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_flags": "0", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_min_size": "0", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pg_num": "32", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_pgp_num": "0", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_size": "2", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_default_type": "replicated", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-13T05:02:05.138 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_push_per_object_cost": "1000", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recover_clone_overlap": "true", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_cost": "20971520", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active": "0", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_max_single_start": "1", 2024-09-13T05:02:05.139 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_op_priority": "3", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_priority": "5", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep": "0.000000", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_repair_during_recovery": "false", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_requested_scrub_priority": "120", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_auto_repair": "false", 2024-09-13T05:02:05.140 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_begin_hour": "0", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_begin_week_day": "0", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_chunk_max": "25", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_chunk_min": "5", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_cost": "52428800", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_during_recovery": "false", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_end_hour": "0", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_end_week_day": "0", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_event_cost": "4096", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-13T05:02:05.141 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_invalid_stats": "true", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_max_preemptions": "5", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_priority": "5", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_sleep": "0.000000", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-13T05:02:05.142 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_skip_data_digest": "false", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_smart_report_timeout": "5", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_cost": "1048576", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_priority": "5", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_target_transaction_size": "30", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-13T05:02:05.143 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_tracing": "false", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_use_stale_snap": "false", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "osdc_blkin_trace_all": "false", 2024-09-13T05:02:05.144 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_kill_at": "0", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_max_join_drift": "10", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_min": "500", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_min_wait": "0.050000", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_propose_interval": "1.000000", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_max": "500", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_service_trim_min": "250", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_stash_full_interval": "25", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_trim_max": "500", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "paxos_trim_min": "250", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "perf": "true", 2024-09-13T05:02:05.145 INFO:teuthology.orchestra.run.smithi053.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "-", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrv": "", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "public_bind_addr": "-", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "public_network": "", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "public_network_interface": "", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "qat_compressor_enabled": "false", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "qat_compressor_session_max_number": "256", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_mon_op_timeout": "0", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_osd_op_timeout": "0", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "rados_tracing": "false", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_atime_update_interval": "60", 2024-09-13T05:02:05.146 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_balance_parent_reads": "false", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_balance_snap_reads": "false", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blkin_trace_all": "false", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache": "true", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_policy": "writearound", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_size": "33554432", 2024-09-13T05:02:05.147 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_clone_copy_on_read": "false", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_compression_hint": "none", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_concurrent_management_ops": "10", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_clone_format": "auto", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_data_pool": "", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_features": "61", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_format": "2", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_map_options": "", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_order": "22", 2024-09-13T05:02:05.148 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_pool": "rbd", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_stripe_count": "0", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_default_stripe_unit": "0", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_enable_alloc_hint": "true", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_io_scheduler": "simple", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-13T05:02:05.149 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_order": "24", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_pool": "", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_journal_splay_width": "4", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_localize_parent_reads": "false", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_localize_snap_reads": "false", 2024-09-13T05:02:05.150 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-13T05:02:05.151 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-13T05:02:05.152 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_mtime_update_interval": "60", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_non_blocking_aio": "true", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_op_thread_timeout": "60", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_op_threads": "1", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_parent_cache_enabled": "false", 2024-09-13T05:02:05.153 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_plugins": "", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_burst": "0", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_bps_limit": "0", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_exclude_ops": "0", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_burst": "0", 2024-09-13T05:02:05.154 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_iops_limit": "0", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-13T05:02:05.155 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_read_from_replica_policy": "default", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_skip_partial_discard": "true", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-13T05:02:05.156 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_tracing": "false", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_validate_names": "true", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd_validate_pool": "true", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "restapi_base_url": "", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "restapi_log_level": "", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_acl_grants_max_num": "100", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_admin_entry": "admin", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_backend_store": "rados", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_barbican_url": "", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_beast_enable_async": "true", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-13T05:02:05.157 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_enabled": "true", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_expiry_interval": "900", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cache_lru_size": "10000", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_config_store": "rados", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_content_length_compat": "false", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_copy_obj_progress": "true", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cors_rules_max_num": "100", 2024-09-13T05:02:05.158 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_cross_domain_policy": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_addr": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_password": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_kmip_username": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_require_ssl": "true", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-13T05:02:05.159 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_addr": "", 2024-09-13T05:02:05.160 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_auth": "token", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_namespace": "", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_prefix": "", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_token_file": "", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_buffersize": "524288", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_low_speed_time": "300", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-13T05:02:05.161 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_changes_size": "1000", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_num_shards": "128", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-13T05:02:05.162 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_log_window": "30", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-13T05:02:05.163 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-13T05:02:05.164 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dns_name": "", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dns_s3website_name": "", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_dynamic_resharding": "true", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_gc_threads": "true", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_lc_threads": "true", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_ops_log": "false", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_quota_threads": "true", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_static_website": "false", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enable_usage_log": "false", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_enforce_swift_acls": "true", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_exit_timeout_secs": "120", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_expose_bucket": "false", 2024-09-13T05:02:05.165 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_extended_http_attrs": "", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_filter": "none", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_frontends": "beast port=7480", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_deferred": "50", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_objs": "32", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_gc_processor_period": "3600", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-13T05:02:05.166 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_init_timeout": "300", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_domain": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_password": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_password_path": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_project": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_tenant": "", 2024-09-13T05:02:05.167 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_token": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_token_path": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_admin_user": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_api_version": "2", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_domain": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_password": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_project": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_barbican_user": "", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-13T05:02:05.168 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_url": "", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_debug_interval": "-1", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_lock_max_time": "90", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_objs": "32", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_rules": "1000", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_worker": "3", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lc_thread_delay": "0", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_dnattr": "uid", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-13T05:02:05.169 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_searchfilter": "", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_http_headers": "", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_log_object_name_utc": "false", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attr_name_len": "0", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attr_size": "0", 2024-09-13T05:02:05.170 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_chunk_size": "4194304", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_listing_results": "1000", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_notify_retries": "10", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_put_param_size": "1048576", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_put_size": "5368709120", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_max_slo_entries": "1000", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_md_log_max_shards": "64", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-13T05:02:05.171 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_mp_lock_max_time": "600", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_max_gc": "300", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-13T05:02:05.172 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_async_rados_threads": "32", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_num_control_oids": "8", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_numa_node": "-1", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_chunk_size": "100", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_gc_interval": "600", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-13T05:02:05.173 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_thread_timeout": "600", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_op_tracing": "false", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_token": "", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_url": "", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_opa_verify_ssl": "true", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_rados": "false", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_ops_log_socket_path": "", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-13T05:02:05.174 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_push_interval": "2.000000", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_print_continue": "true", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rados_tracing": "false", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm": "", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm_id": "", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-13T05:02:05.175 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_region": "", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_request_uri": "", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_batch_size": "64", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_max_aio": "128", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_num_logs": "16", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_reshard_thread_interval": "600", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_resolve_cname": "false", 2024-09-13T05:02:05.176 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_run_sync_thread": "true", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_scheduler_type": "throttler", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_script_uri": "", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_service_provider_name": "", 2024-09-13T05:02:05.177 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_client_id": "", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_client_secret": "", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_entry": "sts", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_key": "sts", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_min_session_duration": "900", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sts_token_introspection_url": "", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_account_in_url": "false", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_auth_entry": "auth", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_auth_url": "", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_custom_header": "", 2024-09-13T05:02:05.178 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_need_stats": "true", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_tenant_name": "", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_token_expiration": "86400", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_url": "", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_url_prefix": "swift", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_lease_period": "120", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-13T05:02:05.179 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_thread_pool_size": "512", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_comment": "", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_createby": "", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_encoding": "", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_flag": "false", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_origin": "", 2024-09-13T05:02:05.180 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_torrent_tracker": "", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_trust_forwarded_https": "false", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_max_shards": "32", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_usage_max_user_shards": "1", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_use_opa_authz": "false", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_max_buckets": "1000", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_policies_max_num": "100", 2024-09-13T05:02:05.181 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_user_unique_email": "true", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_verify_ssl": "true", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone": "", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone_id": "", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup": "", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup_id": "", 2024-09-13T05:02:05.182 INFO:teuthology.orchestra.run.smithi053.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rgwlc_auto_session_clear": "true", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_block_size": "4096", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_size": "536870912", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-13T05:02:05.183 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_index_type": "binary_search", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_partition_filters": "false", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_perf": "false", 2024-09-13T05:02:05.184 INFO:teuthology.orchestra.run.smithi053.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "run_dir": "/var/run/ceph", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_block_create": "true", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_cache_lru_size": "67108864", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_default_max_object_size": "16777216", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_device_size": "53687091200", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_batch_capacity": "16", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-13T05:02:05.185 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_main_device_type": "SSD", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "seastore_segment_size": "67108864", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "set_keepcaps": "false", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "setgroup": "", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "setuser": "", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "setuser_match_path": "", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-13T05:02:05.186 INFO:teuthology.orchestra.run.smithi053.stdout: "thp": "false", 2024-09-13T05:02:05.187 INFO:teuthology.orchestra.run.smithi053.stdout: "threadpool_default_timeout": "60", 2024-09-13T05:02:05.187 INFO:teuthology.orchestra.run.smithi053.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-13T05:02:05.187 INFO:teuthology.orchestra.run.smithi053.stdout: "throttler_perf_counter": "true" 2024-09-13T05:02:05.187 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:05.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-13T05:02:05.305 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a: {} 2024-09-13T05:02:05.305 INFO:teuthology.orchestra.run.smithi053.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:02:05.315 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b: {} 2024-09-13T05:02:05.315 INFO:teuthology.orchestra.run.smithi053.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:02:05.324 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c: {} 2024-09-13T05:02:05.324 INFO:teuthology.orchestra.run.smithi053.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:02:05.334 DEBUG:teuthology.orchestra.run.smithi110:> type ceph-objectstore-tool 2024-09-13T05:02:05.353 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T05:02:05.353 DEBUG:teuthology.orchestra.run.smithi053:> type ceph-objectstore-tool 2024-09-13T05:02:05.368 INFO:teuthology.orchestra.run.smithi053.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-13T05:02:05.368 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-13T05:02:05.379 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-13T05:02:05.379 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-13T05:02:05.379 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:05.380 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:05.380 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-13T05:02:05.380 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-13T05:02:05.380 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-13T05:02:05.381 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-13T05:02:05.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:05.381 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-13T05:02:05.381 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:05.382 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-13T05:02:05.382 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:02:05.382 DEBUG:teuthology.orchestra.run.smithi053:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-13T05:02:05.395 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:05.394+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:05.449 INFO:teuthology.orchestra.run.smithi053.stdout:mapped writes DISABLED 2024-09-13T05:02:05.449 INFO:teuthology.orchestra.run.smithi053.stdout:mapped reads DISABLED 2024-09-13T05:02:05.449 INFO:teuthology.orchestra.run.smithi053.stdout:Seed set to 4847 2024-09-13T05:02:05.483 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:05.482+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:05.533 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:05.534 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:05.534 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:05.534 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:05.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:05.569 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-13T05:02:05.585 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:05.585 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:05.586 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:05.585+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:05.618 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:05.618 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:05.651 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:05.651 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:05.658 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:05.658 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:05.668 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:05.672 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:05.672 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:05.699 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:05.699 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:05.700 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:05.699+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:05.716 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:05.717 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:05.733 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-13T05:02:05.741 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:05.741 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:05.773 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:05.774 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:05.785 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:05.785+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:05.799 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:05.800 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:05.809 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:05.873 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:02:05.886 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:05.885+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:05.890 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-13T05:02:05.964 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:05.964 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:05.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:06.015 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:06.014+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:06.074 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-13T05:02:06.088 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:06.087+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:06.126 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:06.127 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:06.136 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:06.188 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:06.187+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:06.232 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-13T05:02:06.259 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:06.259 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:06.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:06.310 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:06.309+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:06.372 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-13T05:02:06.415 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:06.415+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:06.419 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:06.419 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:06.428 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:06.512 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-13T05:02:06.513 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:06.511+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:06.578 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:06.579 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:06.587 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:06.591 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:06.590+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:06.692 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:06.691+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:06.735 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:06.736 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:06.744 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:06.798 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:06.792+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:06.851 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:06.851 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:06.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:06.902 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:06.901+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:06.995 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:06.995+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:07.007 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.008 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:07.096 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:07.096+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:07.123 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.123 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.132 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:07.197 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:07.196+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:07.247 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.247 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.256 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:07.299 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:07.298+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:07.399 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:07.398+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:07.399 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.400 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:07.500 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:07.499+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.559 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:07.603 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:07.602+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:07.660 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.660 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.669 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:07.709 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:07.709+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:07.789 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:07.805 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:07.804+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:07.808 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.809 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:07.891 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:02:07.932 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:07.933 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:07.933 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:07.932+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:07.950 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:07.950 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:07.963 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:07.963 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:07.966 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:07.966 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:07.972 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:07.983 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:07.984 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:08.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:08.000 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:08.009 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:08.008+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:08.025 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:08.026 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:08.042 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:08.042 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:08.058 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:08.058 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:08.076 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:08.076 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:08.076 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:08.076 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:08.085 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:08.113 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:08.113+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:08.186 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:08.187 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:08.187 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:08.187 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:08.187 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:08.195 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:08.213 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:08.212+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:08.296 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:08.296 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:08.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4235.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.179686+0000", 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.116798113, 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00053168499999999995, 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179686+0000", 2024-09-13T05:02:08.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179686+0000", 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179686+0000", 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179694+0000", 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2830000000000001e-06 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179703+0000", 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8510000000000005e-06 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179711+0000", 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.61e-06 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179740+0000", 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9961e-05 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.179758+0000", 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7943000000000001e-05 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.180191+0000", 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043262699999999999 2024-09-13T05:02:08.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.180218+0000", 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6585e-05 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.301 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:08.305 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:08.349 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:08.348+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:08.407 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:08.407 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:08.408 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:08.416 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:08.413+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:08.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:08.514 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:08.514+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:08.559 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:08.559 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:08.559 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:08.560 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:08.560 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:08.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:08.615 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:08.614+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:35 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.202259+0000", 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.46739309, 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041004850000000001, 2024-09-13T05:02:08.672 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 35 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202259+0000", 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202259+0000", 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202259+0000", 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202308+0000", 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9279e-05 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202314+0000", 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6160000000000001e-06 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202317+0000", 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5540000000000002e-06 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.204553+0000", 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022353020000000002 2024-09-13T05:02:08.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.204632+0000", 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9048999999999995e-05 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.204766+0000", 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013436700000000001 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205803+0000", 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001037156 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206284+0000", 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048067999999999999 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206325+0000", 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1510000000000001e-05 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206359+0000", 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4183999999999997e-05 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:36 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.467329738, 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042082860000000003, 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 36 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:08.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202325+0000", 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2619999999999999e-06 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202329+0000", 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7500000000000001e-06 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202332+0000", 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2600000000000001e-06 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205205+0000", 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028723030000000001 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205253+0000", 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8458000000000003e-05 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205337+0000", 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4499999999999994e-05 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206379+0000", 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010417250000000001 2024-09-13T05:02:08.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206460+0000", 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1100999999999995e-05 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206499+0000", 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8398000000000001e-05 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206531+0000", 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1917e-05 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:37 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.467316122, 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044315279999999997, 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 37 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:08.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202338+0000", 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1210000000000001e-06 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202341+0000", 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2389999999999998e-06 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202344+0000", 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6340000000000001e-06 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205834+0000", 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034905420000000001 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205928+0000", 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3741999999999998e-05 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206061+0000", 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013342599999999999 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206545+0000", 2024-09-13T05:02:08.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048334400000000003 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206699+0000", 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015441399999999999 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206730+0000", 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1143999999999999e-05 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206767+0000", 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7128000000000001e-05 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380124+0000", 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289527873000001, 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041181730000000001, 2024-09-13T05:02:08.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 109 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380124+0000", 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380124+0000", 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380124+0000", 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380167+0000", 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3806e-05 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380176+0000", 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3100000000000001e-06 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380181+0000", 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7030000000000003e-06 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382183+0000", 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020011299999999998 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382230+0000", 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7848e-05 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382310+0000", 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9838000000000004e-05 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384162+0000", 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001852161 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384196+0000", 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3349999999999997e-05 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384222+0000", 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6482000000000002e-05 2024-09-13T05:02:08.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384242+0000", 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0083999999999999e-05 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380202+0000", 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289450412000001, 2024-09-13T05:02:08.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041373750000000004, 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 110 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380202+0000", 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380202+0000", 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380201+0000", 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380207+0000", 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4869999999999999e-06 2024-09-13T05:02:08.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380216+0000", 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2530000000000004e-06 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380221+0000", 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2870000000000004e-06 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382907+0000", 2024-09-13T05:02:08.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002685607 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382962+0000", 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5504000000000003e-05 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383035+0000", 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2608999999999995e-05 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384278+0000", 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012430659999999999 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384294+0000", 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5854000000000001e-05 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384319+0000", 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5174999999999999e-05 2024-09-13T05:02:08.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384339+0000", 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0313e-05 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289424663, 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045436620000000004, 2024-09-13T05:02:08.702 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 111 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380231+0000", 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5870000000000001e-06 2024-09-13T05:02:08.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380237+0000", 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0850000000000002e-06 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380249+0000", 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2037e-05 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383346+0000", 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003096901 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383390+0000", 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3937999999999998e-05 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383468+0000", 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8220000000000007e-05 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384346+0000", 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087789499999999996 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384726+0000", 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000380388 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384751+0000", 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5398999999999999e-05 2024-09-13T05:02:08.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384771+0000", 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9661999999999999e-05 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380256+0000", 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289396292999999, 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0054419129999999996, 2024-09-13T05:02:08.708 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 112 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380256+0000", 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380256+0000", 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380255+0000", 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380305+0000", 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9851000000000001e-05 2024-09-13T05:02:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380314+0000", 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4479999999999994e-06 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380319+0000", 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4489999999999998e-06 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383689+0000", 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033696519999999999 2024-09-13T05:02:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383737+0000", 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7870000000000001e-05 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383820+0000", 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3602000000000004e-05 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385610+0000", 2024-09-13T05:02:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001789674 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385649+0000", 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9461e-05 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385678+0000", 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8773e-05 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385698+0000", 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9664999999999999e-05 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289320019, 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0054681809999999999, 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 113 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380336+0000", 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2230000000000001e-06 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380341+0000", 2024-09-13T05:02:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380352+0000", 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1785e-05 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384363+0000", 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040111089999999997 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384409+0000", 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5799e-05 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384479+0000", 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9468000000000006e-05 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385739+0000", 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012602539999999999 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385745+0000", 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4579999999999998e-06 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385781+0000", 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5728000000000001e-05 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385800+0000", 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9072999999999998e-05 2024-09-13T05:02:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380359+0000", 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289293300000001, 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058786300000000001, 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 114 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380359+0000", 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380359+0000", 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380358+0000", 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380363+0000", 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380375+0000", 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2128e-05 2024-09-13T05:02:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380380+0000", 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8300000000000003e-06 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384798+0000", 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044188609999999996 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384852+0000", 2024-09-13T05:02:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3226000000000002e-05 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384919+0000", 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7354000000000006e-05 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385807+0000", 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00088828300000000002 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386188+0000", 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038091299999999998 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386217+0000", 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8513e-05 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386237+0000", 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0695000000000001e-05 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289262678, 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006229315, 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 115 2024-09-13T05:02:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380431+0000", 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1675000000000003e-05 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380440+0000", 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8910000000000007e-06 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380447+0000", 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7320000000000001e-06 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385156+0000", 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047083309999999996 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385200+0000", 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:02:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385282+0000", 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2164999999999997e-05 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386271+0000", 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00098910800000000004 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386566+0000", 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029463699999999999 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386600+0000", 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3603999999999999e-05 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386619+0000", 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9143999999999998e-05 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380455+0000", 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289197368, 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063589129999999999, 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 116 2024-09-13T05:02:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380455+0000", 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380455+0000", 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380454+0000", 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380458+0000", 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2560000000000004e-06 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380465+0000", 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1940000000000003e-06 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380469+0000", 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9550000000000002e-06 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385828+0000", 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0053589689999999999 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385876+0000", 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8522000000000001e-05 2024-09-13T05:02:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385952+0000", 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5629000000000004e-05 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386627+0000", 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067552300000000003 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386771+0000", 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014388500000000001 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386794+0000", 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3008999999999999e-05 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386814+0000", 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9497999999999999e-05 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 11.289179687000001, 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066318890000000002, 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 117 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380476+0000", 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1130000000000003e-06 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380482+0000", 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9290000000000003e-06 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380486+0000", 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999999e-06 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386288+0000", 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058020939999999998 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386334+0000", 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6069000000000002e-05 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386411+0000", 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6779000000000005e-05 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386820+0000", 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00040942399999999998 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.387063+0000", 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024340000000000001 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.387087+0000", 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4037e-05 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.387104+0000", 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6918999999999998e-05 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779131+0000", 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890521421000001, 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042690239999999997, 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 135 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779131+0000", 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779131+0000", 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779130+0000", 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779142+0000", 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2517999999999999e-05 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779149+0000", 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0319999999999998e-06 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779155+0000", 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3260000000000003e-06 2024-09-13T05:02:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.781636+0000", 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024811550000000001 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.781713+0000", 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7763999999999996e-05 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.781811+0000", 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7412999999999997e-05 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782690+0000", 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087884300000000003 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783317+0000", 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062709199999999999 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783363+0000", 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6310999999999998e-05 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783400+0000", 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6569999999999997e-05 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890490783000001, 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043404610000000003, 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 136 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779165+0000", 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7620000000000001e-06 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779171+0000", 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5150000000000003e-06 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779176+0000", 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8289999999999998e-06 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782109+0000", 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029328179999999998 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782161+0000", 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782230+0000", 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9307999999999994e-05 2024-09-13T05:02:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783412+0000", 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011818309999999999 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783450+0000", 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8118000000000003e-05 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783482+0000", 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2329000000000002e-05 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783502+0000", 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9522e-05 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890471121999999, 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051891749999999999, 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 137 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779224+0000", 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3047000000000003e-05 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779229+0000", 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3260000000000003e-06 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779232+0000", 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.219e-06 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782715+0000", 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003482855 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782764+0000", 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8566999999999998e-05 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782848+0000", 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4756e-05 2024-09-13T05:02:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784277+0000", 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001428971 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784286+0000", 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7099999999999996e-06 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784333+0000", 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6882000000000003e-05 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784370+0000", 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7175000000000002e-05 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890415939, 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063116589999999998, 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 138 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779238+0000", 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1940000000000001e-06 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779241+0000", 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.157e-06 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779244+0000", 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783534+0000", 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042906410000000004 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783597+0000", 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3026000000000003e-05 2024-09-13T05:02:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783669+0000", 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1044999999999995e-05 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785428+0000", 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001759622 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785471+0000", 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3305e-05 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785513+0000", 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1820000000000003e-05 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785548+0000", 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4535000000000001e-05 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890405777, 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063959350000000002, 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 139 2024-09-13T05:02:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779248+0000", 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1890000000000001e-06 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779253+0000", 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6489999999999999e-06 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779256+0000", 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.88e-06 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783899+0000", 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046426979999999998 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783950+0000", 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1665000000000001e-05 2024-09-13T05:02:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784019+0000", 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9187e-05 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785588+0000", 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015683610000000001 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785593+0000", 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.916e-06 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785623+0000", 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0128999999999999e-05 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785642+0000", 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9471e-05 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779261+0000", 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890391427000001, 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064895719999999999, 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 140 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779261+0000", 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779261+0000", 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779260+0000", 2024-09-13T05:02:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779310+0000", 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9830999999999998e-05 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779317+0000", 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0849999999999999e-06 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779321+0000", 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2820000000000001e-06 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784408+0000", 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050876330000000003 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784483+0000", 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4759999999999996e-05 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784606+0000", 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000122617 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785648+0000", 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001042304 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785682+0000", 2024-09-13T05:02:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.358e-05 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785719+0000", 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7645999999999998e-05 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785750+0000", 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1053000000000002e-05 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779325+0000", 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.890327135, 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066314759999999999, 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 141 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779325+0000", 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779325+0000", 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779324+0000", 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779328+0000", 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4960000000000001e-06 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779334+0000", 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6500000000000001e-06 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779340+0000", 2024-09-13T05:02:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0510000000000002e-06 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785069+0000", 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057296600000000001 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785113+0000", 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3838999999999998e-05 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785186+0000", 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2831999999999997e-05 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785760+0000", 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057425800000000002 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785886+0000", 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012604099999999999 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785922+0000", 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5710999999999997e-05 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785956+0000", 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4397000000000003e-05 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:58.289031+0000", 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.380621552999999, 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039733310000000001, 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 159 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.289031+0000", 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.289031+0000", 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.289030+0000", 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.289033+0000", 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3120000000000001e-06 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.289036+0000", 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6660000000000001e-06 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.289039+0000", 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.292174+0000", 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031352939999999998 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.292224+0000", 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9524999999999998e-05 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.292316+0000", 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2707999999999994e-05 2024-09-13T05:02:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.292814+0000", 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049793700000000005 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.292964+0000", 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014948500000000001 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.292987+0000", 2024-09-13T05:02:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2782e-05 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:58.293004+0000", 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7269e-05 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:08.792 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:08.793 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:08.794 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:08.792+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:08.883 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:02:08.894 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:08.893+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:08.908 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:08.909 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:08.909 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:08.909 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:08.918 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:08.996 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:08.995+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:09.064 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:09.064 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:09.065 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:09.065 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:09.065 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:09.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:09.097 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:09.096+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4235.0:5 1.2 e16/14 1:5cea7035:::rbd_info:head v 16'1, mlcod=0'0)", 2024-09-13T05:02:09.182 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.893059786, 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084071500000000002, 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:09.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287676+0000", 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6070000000000008e-06 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287692+0000", 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6341000000000002e-05 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287705+0000", 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2951e-05 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287772+0000", 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7366000000000005e-05 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287804+0000", 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1602e-05 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.288474+0000", 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067043599999999995 2024-09-13T05:02:09.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.288508+0000", 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3657e-05 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4235.0:7 1.2 e17/14 1:5cea7035:::rbd_info:head v 17'2, mlcod=0'0)", 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.9840078099999996, 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091731799999999997, 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196726+0000", 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1980000000000004e-06 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196736+0000", 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0231000000000001e-05 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196744+0000", 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0229999999999992e-06 2024-09-13T05:02:09.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196803+0000", 2024-09-13T05:02:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9004e-05 2024-09-13T05:02:09.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196833+0000", 2024-09-13T05:02:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.94e-05 2024-09-13T05:02:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.197605+0000", 2024-09-13T05:02:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077226900000000002 2024-09-13T05:02:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.197636+0000", 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1417000000000001e-05 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.888870+0000", 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.29185746200000001, 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003860437, 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888870+0000", 2024-09-13T05:02:09.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888870+0000", 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888867+0000", 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888880+0000", 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2403e-05 2024-09-13T05:02:09.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888903+0000", 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3133e-05 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888916+0000", 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888967+0000", 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1168000000000002e-05 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:09.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888972+0000", 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5809999999999998e-06 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.891272+0000", 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022999119999999999 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.891409+0000", 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137491 2024-09-13T05:02:09.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.891619+0000", 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021031000000000001 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892622+0000", 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010026919999999999 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892641+0000", 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9045000000000001e-05 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892691+0000", 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-05 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892730+0000", 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9223e-05 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:11 3.1f e22/19 ondisk, result = 0)", 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.892522+0000", 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28820473400000002, 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105386, 2024-09-13T05:02:09.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892522+0000", 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892522+0000", 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892521+0000", 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892527+0000", 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892538+0000", 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.028e-05 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892547+0000", 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1929999999999999e-06 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892606+0000", 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8746999999999998e-05 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892614+0000", 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5599999999999994e-06 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892628+0000", 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3485999999999999e-05 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.208 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:12 3.1 3:86cf5969:::rbd_header.117c773772bd:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.892891+0000", 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28783574699999998, 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011613470000000001, 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 12 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892891+0000", 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892891+0000", 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892890+0000", 2024-09-13T05:02:09.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892896+0000", 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9800000000000003e-06 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892906+0000", 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9920000000000002e-06 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892911+0000", 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.916e-06 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892962+0000", 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0921000000000003e-05 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893101+0000", 2024-09-13T05:02:09.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000138651 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893337+0000", 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023641999999999999 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893952+0000", 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061527900000000004 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893963+0000", 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.894010+0000", 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7679000000000002e-05 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.894053+0000", 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2191000000000003e-05 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:12 3.1 e22/19 ondisk, result = 0)", 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.893888+0000", 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28683916500000001, 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7738999999999996e-05, 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.215 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893888+0000", 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893888+0000", 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893887+0000", 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893891+0000", 2024-09-13T05:02:09.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1969999999999998e-06 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893901+0000", 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5030000000000003e-06 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893905+0000", 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.301e-06 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893940+0000", 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4476e-05 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893947+0000", 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3690000000000001e-06 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893956+0000", 2024-09-13T05:02:09.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6649999999999992e-06 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.898546+0000", 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.282181405, 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028609399999999998, 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898546+0000", 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898546+0000", 2024-09-13T05:02:09.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898543+0000", 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898560+0000", 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7146000000000002e-05 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898584+0000", 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4338999999999999e-05 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898596+0000", 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1292e-05 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898658+0000", 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2566999999999999e-05 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898755+0000", 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7054999999999998e-05 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898832+0000", 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6481000000000002e-05 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.898943+0000", 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28178401400000003, 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029036300000000003, 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.225 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898943+0000", 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898943+0000", 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898942+0000", 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898948+0000", 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3899999999999998e-06 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898961+0000", 2024-09-13T05:02:09.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2622e-05 2024-09-13T05:02:09.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898969+0000", 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899012+0000", 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3007999999999999e-05 2024-09-13T05:02:09.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899062+0000", 2024-09-13T05:02:09.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9966000000000002e-05 2024-09-13T05:02:09.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899233+0000", 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017133 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.232 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:16 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.232 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.899346+0000", 2024-09-13T05:02:09.232 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28138080799999998, 2024-09-13T05:02:09.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000235956, 2024-09-13T05:02:09.232 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.233 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.233 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.233 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 16 2024-09-13T05:02:09.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.234 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899346+0000", 2024-09-13T05:02:09.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899346+0000", 2024-09-13T05:02:09.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899345+0000", 2024-09-13T05:02:09.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899352+0000", 2024-09-13T05:02:09.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3470000000000004e-06 2024-09-13T05:02:09.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899362+0000", 2024-09-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7659999999999993e-06 2024-09-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899367+0000", 2024-09-13T05:02:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1449999999999999e-06 2024-09-13T05:02:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899400+0000", 2024-09-13T05:02:09.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3033e-05 2024-09-13T05:02:09.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899446+0000", 2024-09-13T05:02:09.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6174999999999998e-05 2024-09-13T05:02:09.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899582+0000", 2024-09-13T05:02:09.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013624199999999999 2024-09-13T05:02:09.241 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.242 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:17 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.242 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:09.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28105967700000001, 2024-09-13T05:02:09.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015649500000000001, 2024-09-13T05:02:09.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.244 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.244 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17 2024-09-13T05:02:09.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:09.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:09.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:09.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899671+0000", 2024-09-13T05:02:09.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6959999999999996e-06 2024-09-13T05:02:09.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899680+0000", 2024-09-13T05:02:09.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2460000000000003e-06 2024-09-13T05:02:09.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899684+0000", 2024-09-13T05:02:09.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.941e-06 2024-09-13T05:02:09.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899710+0000", 2024-09-13T05:02:09.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6392000000000001e-05 2024-09-13T05:02:09.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899750+0000", 2024-09-13T05:02:09.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0522999999999998e-05 2024-09-13T05:02:09.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899824+0000", 2024-09-13T05:02:09.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3324999999999993e-05 2024-09-13T05:02:09.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.252 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.252 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:18 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28082768699999999, 2024-09-13T05:02:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016291299999999999, 2024-09-13T05:02:09.253 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18 2024-09-13T05:02:09.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.255 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:09.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:09.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:09.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899903+0000", 2024-09-13T05:02:09.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1930000000000003e-06 2024-09-13T05:02:09.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899911+0000", 2024-09-13T05:02:09.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8290000000000008e-06 2024-09-13T05:02:09.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899915+0000", 2024-09-13T05:02:09.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9199999999999997e-06 2024-09-13T05:02:09.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899940+0000", 2024-09-13T05:02:09.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5398999999999999e-05 2024-09-13T05:02:09.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899983+0000", 2024-09-13T05:02:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2531000000000003e-05 2024-09-13T05:02:09.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900062+0000", 2024-09-13T05:02:09.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9677000000000003e-05 2024-09-13T05:02:09.263 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.264 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.264 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.264 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:19 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.264 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.900149+0000", 2024-09-13T05:02:09.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.280578043, 2024-09-13T05:02:09.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014279900000000001, 2024-09-13T05:02:09.265 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.265 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.265 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19 2024-09-13T05:02:09.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900149+0000", 2024-09-13T05:02:09.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900149+0000", 2024-09-13T05:02:09.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900148+0000", 2024-09-13T05:02:09.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900153+0000", 2024-09-13T05:02:09.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9389999999999996e-06 2024-09-13T05:02:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900162+0000", 2024-09-13T05:02:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5310000000000001e-06 2024-09-13T05:02:09.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900166+0000", 2024-09-13T05:02:09.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000003e-06 2024-09-13T05:02:09.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900191+0000", 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5307e-05 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900230+0000", 2024-09-13T05:02:09.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8770999999999999e-05 2024-09-13T05:02:09.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900292+0000", 2024-09-13T05:02:09.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1852000000000003e-05 2024-09-13T05:02:09.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.274 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.274 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.274 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:20 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.900645+0000", 2024-09-13T05:02:09.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.28008242500000002, 2024-09-13T05:02:09.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032938800000000001, 2024-09-13T05:02:09.275 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.275 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.275 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900645+0000", 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900645+0000", 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900644+0000", 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900648+0000", 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.515e-06 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900658+0000", 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1600000000000004e-06 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900662+0000", 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900689+0000", 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6472e-05 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900734+0000", 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5352000000000002e-05 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900974+0000", 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000239812 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.280 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:21 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.901066+0000", 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.279661469, 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000233494, 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 21 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901066+0000", 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901066+0000", 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901065+0000", 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901070+0000", 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3720000000000004e-06 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901079+0000", 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2629999999999992e-06 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901084+0000", 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6929999999999998e-06 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901117+0000", 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2628e-05 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901161+0000", 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4267999999999997e-05 2024-09-13T05:02:09.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901299+0000", 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013798099999999999 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:22 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.285 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.27934752499999999, 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014782700000000001, 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22 2024-09-13T05:02:09.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.287 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:09.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901383+0000", 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.794e-06 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901390+0000", 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.807e-06 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901394+0000", 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8010000000000001e-06 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901419+0000", 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5159000000000001e-05 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901457+0000", 2024-09-13T05:02:09.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7589000000000002e-05 2024-09-13T05:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901527+0000", 2024-09-13T05:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0290999999999995e-05 2024-09-13T05:02:09.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:24 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.902237+0000", 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.27848986100000001, 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016992799999999999, 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 24 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902237+0000", 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902237+0000", 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902236+0000", 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902241+0000", 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2729999999999996e-06 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902250+0000", 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9390000000000003e-06 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902254+0000", 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1289999999999998e-06 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902289+0000", 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5317000000000001e-05 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902337+0000", 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8010000000000003e-05 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902407+0000", 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9931999999999994e-05 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.296 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:25 3.1 3:86cf5969:::rbd_header.117c773772bd:head [watch watch cookie 140166391944464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.902537+0000", 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.278190041, 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000975175, 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.297 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 25 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902537+0000", 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902537+0000", 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902536+0000", 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902541+0000", 2024-09-13T05:02:09.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.566e-06 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902550+0000", 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7350000000000002e-06 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902560+0000", 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.01e-05 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902591+0000", 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1683000000000001e-05 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902638+0000", 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6592999999999999e-05 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902725+0000", 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7255000000000004e-05 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903265+0000", 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054003199999999997 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903435+0000", 2024-09-13T05:02:09.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016971099999999999 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903460+0000", 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5174e-05 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903512+0000", 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1952e-05 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:25 3.1 e22/19 ondisk, result = 0)", 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.903385+0000", 2024-09-13T05:02:09.304 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.27734203600000001, 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013565800000000001, 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903385+0000", 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903385+0000", 2024-09-13T05:02:09.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903384+0000", 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903391+0000", 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9829999999999999e-06 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903400+0000", 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4950000000000008e-06 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903405+0000", 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0629999999999996e-06 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903427+0000", 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2419999999999999e-05 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903432+0000", 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7679999999999999e-06 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903521+0000", 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8727000000000006e-05 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.309 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:09.310 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:09.309+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:09.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:09.411 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:09.411+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:09.419 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:09.419 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:09.419 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:09.420 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:09.429 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:09.512 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:09.512+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:09.576 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:09.576 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:09.576 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:09.576 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:09.576 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:09.585 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:09.613 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:09.613+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:09.714 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:09.714+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4235.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:09.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.179043+0000", 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.552546444000001, 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001385266, 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4235", 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2115106039", 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179043+0000", 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179043+0000", 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179041+0000", 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179052+0000", 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1083e-05 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179067+0000", 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4440000000000001e-05 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179082+0000", 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5537e-05 2024-09-13T05:02:09.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179165+0000", 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2551000000000003e-05 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179384+0000", 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021954400000000001 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6", 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179542+0000", 2024-09-13T05:02:09.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015789999999999999 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180067+0000", 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052489400000000003 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180347+0000", 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027953800000000002 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180379+0000", 2024-09-13T05:02:09.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2577999999999999e-05 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180428+0000", 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9308999999999998e-05 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:16 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'6, mlcod=15'4)", 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.190612+0000", 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.540978151999999, 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097868, 2024-09-13T05:02:09.739 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190612+0000", 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190612+0000", 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190610+0000", 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190625+0000", 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5559000000000001e-05 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190654+0000", 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8487e-05 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190666+0000", 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2023e-05 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190765+0000", 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9468000000000004e-05 2024-09-13T05:02:09.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.190834+0000", 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8835000000000001e-05 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.191554+0000", 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071971899999999996 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.191590+0000", 2024-09-13T05:02:09.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6519000000000001e-05 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:20 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'10, mlcod=15'8)", 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.195103+0000", 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.536486452, 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001412188, 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195103+0000", 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195103+0000", 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195102+0000", 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195214+0000", 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111531 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195227+0000", 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3400999999999999e-05 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195233+0000", 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3480000000000003e-06 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195265+0000", 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1983000000000002e-05 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195281+0000", 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6218e-05 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196474+0000", 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001193127 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196515+0000", 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1440000000000003e-05 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:21 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.195632+0000", 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.535958001000001, 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098342500000000001, 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195632+0000", 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195632+0000", 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195631+0000", 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195693+0000", 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2576999999999994e-05 2024-09-13T05:02:09.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195706+0000", 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3178e-05 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195713+0000", 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2589999999999997e-06 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195752+0000", 2024-09-13T05:02:09.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9752999999999997e-05 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195777+0000", 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4584999999999999e-05 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196582+0000", 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080502500000000001 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196615+0000", 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3161999999999999e-05 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:31 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'17, mlcod=15'14)", 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.202696+0000", 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.528893627, 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010801249999999999, 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.753 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202696+0000", 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202696+0000", 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202695+0000", 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202712+0000", 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6399999999999999e-05 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202720+0000", 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.909e-06 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202724+0000", 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8480000000000003e-06 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202763+0000", 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9631000000000001e-05 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202788+0000", 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5239e-05 2024-09-13T05:02:09.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203741+0000", 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095256099999999999 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203776+0000", 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5221e-05 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:32 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'18, mlcod=15'14)", 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.203142+0000", 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.528448099, 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001006875, 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203142+0000", 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203142+0000", 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203141+0000", 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203187+0000", 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6737000000000003e-05 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203198+0000", 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0104e-05 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203202+0000", 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2749999999999997e-06 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203273+0000", 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0753999999999994e-05 2024-09-13T05:02:09.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203296+0000", 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2983999999999999e-05 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204112+0000", 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081669499999999999 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204148+0000", 2024-09-13T05:02:09.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6124000000000002e-05 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:35 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'17)", 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.204847+0000", 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.526742706, 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010255100000000001, 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204847+0000", 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204847+0000", 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204846+0000", 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204918+0000", 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1952000000000005e-05 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204929+0000", 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0489000000000001e-05 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204933+0000", 2024-09-13T05:02:09.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3610000000000003e-06 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204971+0000", 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8037000000000002e-05 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204991+0000", 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0259e-05 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.205830+0000", 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083891599999999999 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.205872+0000", 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2073000000000002e-05 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:38 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'24, mlcod=15'22)", 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.207295+0000", 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.524294958, 2024-09-13T05:02:09.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010032730000000001, 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207295+0000", 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207295+0000", 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207294+0000", 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207343+0000", 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8736999999999998e-05 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207354+0000", 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1741e-05 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207359+0000", 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7979999999999996e-06 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207398+0000", 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9048999999999998e-05 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207420+0000", 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1886999999999999e-05 2024-09-13T05:02:09.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.208265+0000", 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084525099999999999 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.208298+0000", 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2564999999999997e-05 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:54 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'32, mlcod=15'29)", 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.218237+0000", 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.513352580999999, 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010037360000000001, 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218237+0000", 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218237+0000", 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218236+0000", 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218242+0000", 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2859999999999999e-06 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218250+0000", 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218264+0000", 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3681e-05 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218302+0000", 2024-09-13T05:02:09.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8099999999999998e-05 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218325+0000", 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3263000000000001e-05 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.219205+0000", 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088005600000000005 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.219241+0000", 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5632e-05 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:55 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'33, mlcod=15'29)", 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.218700+0000", 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.512889778, 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097018099999999997, 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218700+0000", 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218700+0000", 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218699+0000", 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218765+0000", 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218775+0000", 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.781e-06 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218791+0000", 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5838999999999999e-05 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218826+0000", 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5284999999999998e-05 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218847+0000", 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1540999999999999e-05 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.219639+0000", 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00079182500000000002 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.219670+0000", 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0889999999999997e-05 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.779 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:57 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'35, mlcod=15'33)", 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.220620+0000", 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.510969340999999, 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174504, 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220620+0000", 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220620+0000", 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220613+0000", 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999924 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220677+0000", 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4021000000000002e-05 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220688+0000", 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1399e-05 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220693+0000", 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7129999999999996e-06 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220727+0000", 2024-09-13T05:02:09.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4649e-05 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220748+0000", 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0452e-05 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.221747+0000", 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099969899999999994 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.221795+0000", 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7345999999999997e-05 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:68 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'42, mlcod=15'39)", 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.227836+0000", 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.503753707, 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099165400000000002, 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227836+0000", 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227836+0000", 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227835+0000", 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227842+0000", 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4389999999999997e-06 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227855+0000", 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3162e-05 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227861+0000", 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2960000000000004e-06 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227891+0000", 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0266000000000001e-05 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227911+0000", 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9704e-05 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228769+0000", 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085764800000000003 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228828+0000", 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8909000000000001e-05 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:69 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'43, mlcod=15'39)", 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.228273+0000", 2024-09-13T05:02:09.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.503317052, 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096618300000000002, 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228273+0000", 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228273+0000", 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228272+0000", 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228327+0000", 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5544999999999998e-05 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228340+0000", 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2153e-05 2024-09-13T05:02:09.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228346+0000", 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.426e-06 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228375+0000", 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8866000000000001e-05 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228388+0000", 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3203999999999999e-05 2024-09-13T05:02:09.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.229175+0000", 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00078698400000000001 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.229239+0000", 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3789999999999997e-05 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:71 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'45, mlcod=15'43)", 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.230089+0000", 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.501500975000001, 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010190189999999999, 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230089+0000", 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230089+0000", 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230088+0000", 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230146+0000", 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8363999999999999e-05 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230158+0000", 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1403000000000001e-05 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230163+0000", 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230206+0000", 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3541999999999997e-05 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230228+0000", 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1994999999999999e-05 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231050+0000", 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082189800000000005 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231108+0000", 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7670000000000002e-05 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.796 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:72 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'46, mlcod=15'43)", 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.230536+0000", 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.501053695, 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001235676, 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230536+0000", 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230536+0000", 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230535+0000", 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230607+0000", 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1557999999999995e-05 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230613+0000", 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.353e-06 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230617+0000", 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5530000000000001e-06 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230661+0000", 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3667000000000001e-05 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230679+0000", 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8603000000000001e-05 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231743+0000", 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001063949 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231772+0000", 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8461000000000001e-05 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:86 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'56, mlcod=15'52)", 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.238845+0000", 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.492744203000001, 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00097329100000000004, 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238845+0000", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238845+0000", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238844+0000", 2024-09-13T05:02:09.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238851+0000", 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8589999999999999e-06 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238861+0000", 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2909999999999996e-06 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238868+0000", 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8990000000000004e-06 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238902+0000", 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4057000000000003e-05 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.238916+0000", 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4565e-05 2024-09-13T05:02:09.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239785+0000", 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086894700000000001 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239819+0000", 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3640000000000003e-05 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:87 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'52)", 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.492352318, 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00107336, 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239297+0000", 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9759999999999997e-05 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239307+0000", 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0631e-05 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239314+0000", 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5799999999999997e-06 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239347+0000", 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2789000000000001e-05 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239359+0000", 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2469999999999999e-05 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.240274+0000", 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091490799999999998 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.240311+0000", 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6806e-05 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.809 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:105 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'67, mlcod=15'64)", 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.351215239, 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010332659999999999, 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:09.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380379+0000", 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6380000000000003e-06 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380388+0000", 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3319999999999998e-06 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380392+0000", 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7139999999999999e-06 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380426+0000", 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4817000000000002e-05 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380451+0000", 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4533e-05 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.381384+0000", 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093344999999999995 2024-09-13T05:02:09.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.381408+0000", 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3302000000000002e-05 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:109 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'71, mlcod=15'66)", 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.382395+0000", 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.349194487, 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098331999999999998, 2024-09-13T05:02:09.814 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382395+0000", 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382395+0000", 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382395+0000", 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382455+0000", 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0095e-05 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382462+0000", 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3849999999999996e-06 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382466+0000", 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3689999999999998e-06 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382485+0000", 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9575000000000001e-05 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.382497+0000", 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1409e-05 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.383360+0000", 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086334900000000002 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.383378+0000", 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8529000000000002e-05 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.818 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:134 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'92, mlcod=15'87)", 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.781140+0000", 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.950449298000001, 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096882700000000001, 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781140+0000", 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:09.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781140+0000", 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781140+0000", 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781217+0000", 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7115999999999998e-05 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781226+0000", 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4499999999999993e-06 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781230+0000", 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0289999999999999e-06 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781254+0000", 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3482e-05 2024-09-13T05:02:09.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.781265+0000", 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1232e-05 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.782092+0000", 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082695399999999997 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.782109+0000", 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7091e-05 2024-09-13T05:02:09.822 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:09.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:09.823 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:09.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:09.825 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:09.823+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:09.925 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:09.925+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:09.973 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:09.973 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:09.973 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:09.973 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:09.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:10.067 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:10.066+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:10.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:10.127 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:10.126+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:10.130 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:10.131 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:10.131 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:10.140 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:10.210 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:10.210 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:10.227 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:10.227+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:10.243 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:10.244 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:10.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:10.261 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:10.270 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:10.277 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:10.277 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:10.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:10.294 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:10.294 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:10.311 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:10.312 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:10.328 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:10.327+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:10.328 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:10.329 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:10.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:10.345 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:10.388 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:10.388 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:10.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:10.438 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:10.438+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:10.529 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:10.529+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:10.545 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:10.546 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:10.558 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:10.630 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:10.629+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:1 write 0x9f7a7b2 thru 0x9f7aab0 (0x2ff bytes) 2024-09-13T05:02:10.634 INFO:teuthology.orchestra.run.smithi053.stdout:2 punch from 0x7a1d218 to 0x7a24b81, (0x7969 bytes) 2024-09-13T05:02:10.634 INFO:teuthology.orchestra.run.smithi053.stdout:3 punch from 0x1435edd to 0x143b5ed, (0x5710 bytes) 2024-09-13T05:02:10.634 INFO:teuthology.orchestra.run.smithi053.stdout:4 read 0x8098c88 thru 0x80a8031 (0xf3aa bytes) 2024-09-13T05:02:10.634 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.635 INFO:teuthology.orchestra.run.smithi053.stdout:truncating to largest ever: 0xb420cf6 2024-09-13T05:02:10.635 INFO:teuthology.orchestra.run.smithi053.stdout:6 trunc from 0x9f7aab1 to 0xb420cf6 2024-09-13T05:02:10.643 INFO:teuthology.orchestra.run.smithi053.stdout:7 punch from 0x3793e33 to 0x3799ded, (0x5fba bytes) 2024-09-13T05:02:10.643 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.643 INFO:teuthology.orchestra.run.smithi053.stdout:9 punch from 0x15c5f0e to 0x15cace4, (0x4dd6 bytes) 2024-09-13T05:02:10.643 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.643 INFO:teuthology.orchestra.run.smithi053.stdout:11 punch from 0x9b47bc6 to 0x9b51334, (0x976e bytes) 2024-09-13T05:02:10.643 INFO:teuthology.orchestra.run.smithi053.stdout:truncating to largest ever: 0xcfb254e 2024-09-13T05:02:10.644 INFO:teuthology.orchestra.run.smithi053.stdout:13 trunc from 0xb420cf6 to 0xcfb254e 2024-09-13T05:02:10.650 INFO:teuthology.orchestra.run.smithi053.stdout:15 punch from 0x8ffa1a8 to 0x900751f, (0xd377 bytes) 2024-09-13T05:02:10.650 INFO:teuthology.orchestra.run.smithi053.stdout:16 punch from 0xa98dddc to 0xa998a86, (0xacaa bytes) 2024-09-13T05:02:10.650 INFO:teuthology.orchestra.run.smithi053.stdout:truncating to largest ever: 0xd947bc2 2024-09-13T05:02:10.651 INFO:teuthology.orchestra.run.smithi053.stdout:17 trunc from 0xcfb254e to 0xd947bc2 2024-09-13T05:02:10.656 INFO:teuthology.orchestra.run.smithi053.stdout:truncating to largest ever: 0xe7e5850 2024-09-13T05:02:10.657 INFO:teuthology.orchestra.run.smithi053.stdout:18 trunc from 0xd947bc2 to 0xe7e5850 2024-09-13T05:02:10.664 INFO:teuthology.orchestra.run.smithi053.stdout:19 write 0x114271 thru 0x117f11 (0x3ca1 bytes) 2024-09-13T05:02:10.677 INFO:teuthology.orchestra.run.smithi053.stdout:20 trunc from 0xe7e5850 to 0x58b8754 2024-09-13T05:02:10.680 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:10.680 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:10.688 INFO:teuthology.orchestra.run.smithi053.stdout:22 punch from 0x39bc967 to 0x39c376f, (0x6e08 bytes) 2024-09-13T05:02:10.688 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.690 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:10.698 INFO:teuthology.orchestra.run.smithi053.stdout:25 write 0x96eb0f5 thru 0x96ec203 (0x110f bytes) 2024-09-13T05:02:10.702 INFO:teuthology.orchestra.run.smithi053.stdout:28 read 0x417abc5 thru 0x4186a35 (0xbe71 bytes) 2024-09-13T05:02:10.702 INFO:teuthology.orchestra.run.smithi053.stdout:29 read 0x895dd1e thru 0x896157e (0x3861 bytes) 2024-09-13T05:02:10.702 INFO:teuthology.orchestra.run.smithi053.stdout:32 punch from 0x31778fe to 0x3184bda, (0xd2dc bytes) 2024-09-13T05:02:10.702 INFO:teuthology.orchestra.run.smithi053.stdout:34 write 0x2e64174 thru 0x2e6b2f8 (0x7185 bytes) 2024-09-13T05:02:10.705 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.705 INFO:teuthology.orchestra.run.smithi053.stdout:37 read 0x5f1aa6b thru 0x5f1aebf (0x455 bytes) 2024-09-13T05:02:10.711 INFO:teuthology.orchestra.run.smithi053.stdout:40 writesame 0xa5bd50d thru 0xa5bf2ca data_size 0x36(0x1dbe bytes) 2024-09-13T05:02:10.711 INFO:teuthology.orchestra.run.smithi053.stdout:41 read 0x46cd66 thru 0x47c3d2 (0xf66d bytes) 2024-09-13T05:02:10.721 INFO:teuthology.orchestra.run.smithi053.stdout:42 write 0xe016b6f thru 0xe017fd1 (0x1463 bytes) 2024-09-13T05:02:10.724 INFO:teuthology.orchestra.run.smithi053.stdout:44 writesame 0x36efcc9 thru 0x36fe1ec data_size 0x46(0xe524 bytes) 2024-09-13T05:02:10.727 INFO:teuthology.orchestra.run.smithi053.stdout:45 write 0x579bc1f thru 0x57a2dc6 (0x71a8 bytes) 2024-09-13T05:02:10.730 INFO:teuthology.orchestra.run.smithi053.stdout:46 writesame 0x14ea147 thru 0x14eb902 data_size 0x1f(0x17bc bytes) 2024-09-13T05:02:10.733 INFO:teuthology.orchestra.run.smithi053.stdout:47 writesame 0x2d63aa8 thru 0x2d644af data_size 0xc(0xa08 bytes) 2024-09-13T05:02:10.735 INFO:teuthology.orchestra.run.smithi053.stdout:48 write 0x7f8d2b1 thru 0x7f99d11 (0xca61 bytes) 2024-09-13T05:02:10.735 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:10.734+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:10.738 INFO:teuthology.orchestra.run.smithi053.stdout:49 writesame 0x72994de thru 0x729b0c3 data_size 0x1(0x1be6 bytes) 2024-09-13T05:02:10.743 INFO:teuthology.orchestra.run.smithi053.stdout:51 read 0x87d427a thru 0x87de70e (0xa495 bytes) 2024-09-13T05:02:10.743 INFO:teuthology.orchestra.run.smithi053.stdout:52 punch from 0x2a4043a to 0x2a4b3cd, (0xaf93 bytes) 2024-09-13T05:02:10.743 INFO:teuthology.orchestra.run.smithi053.stdout:53 read 0xd903f19 thru 0xd90cf80 (0x9068 bytes) 2024-09-13T05:02:10.744 INFO:teuthology.orchestra.run.smithi053.stdout:54 punch from 0xb803a05 to 0xb813357, (0xf952 bytes) 2024-09-13T05:02:10.744 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.744 INFO:teuthology.orchestra.run.smithi053.stdout:56 writesame 0xd56725f thru 0xd571d60 data_size 0xe(0xab02 bytes) 2024-09-13T05:02:10.747 INFO:teuthology.orchestra.run.smithi053.stdout:57 punch from 0x31e349c to 0x31e9c29, (0x678d bytes) 2024-09-13T05:02:10.747 INFO:teuthology.orchestra.run.smithi053.stdout:58 read 0xda8b2d8 thru 0xda8e575 (0x329e bytes) 2024-09-13T05:02:10.747 INFO:teuthology.orchestra.run.smithi053.stdout:59 write 0x59c99 thru 0x5b67f (0x19e7 bytes) 2024-09-13T05:02:10.748 INFO:teuthology.orchestra.run.smithi053.stdout:60 read 0x38dc4d9 thru 0x38e6aab (0xa5d3 bytes) 2024-09-13T05:02:10.749 INFO:teuthology.orchestra.run.smithi053.stdout:61 write 0x583d5a5 thru 0x584664e (0x90aa bytes) 2024-09-13T05:02:10.752 INFO:teuthology.orchestra.run.smithi053.stdout:62 trunc from 0xe017fd2 to 0xe639130 2024-09-13T05:02:10.757 INFO:teuthology.orchestra.run.smithi053.stdout:63 read 0xd7b85cb thru 0xd7bb8c3 (0x32f9 bytes) 2024-09-13T05:02:10.757 INFO:teuthology.orchestra.run.smithi053.stdout:68 punch from 0x65fd2ae to 0x6608748, (0xb49a bytes) 2024-09-13T05:02:10.773 INFO:teuthology.orchestra.run.smithi053.stdout:69 trunc from 0xe639130 to 0x39293 2024-09-13T05:02:10.799 INFO:teuthology.orchestra.run.smithi053.stdout:70 write 0x71730c4 thru 0x717fa87 (0xc9c4 bytes) 2024-09-13T05:02:10.803 INFO:teuthology.orchestra.run.smithi053.stdout:71 trunc from 0x717fa88 to 0x5be8df0 2024-09-13T05:02:10.805 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:10.805 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:10.812 INFO:teuthology.orchestra.run.smithi053.stdout:72 read 0xe5dfc6 thru 0xe6b481 (0xd4bc bytes) 2024-09-13T05:02:10.814 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:10.824 INFO:teuthology.orchestra.run.smithi053.stdout:73 write 0xc36a08d thru 0xc3779d7 (0xd94b bytes) 2024-09-13T05:02:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:75 punch from 0x9cf038e to 0x9cf3895, (0x3507 bytes) 2024-09-13T05:02:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:76 read 0x89e5ab0 thru 0x89e80c3 (0x2614 bytes) 2024-09-13T05:02:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:77 read 0x80fda16 thru 0x810d110 (0xf6fb bytes) 2024-09-13T05:02:10.828 INFO:teuthology.orchestra.run.smithi053.stdout:79 write 0x3ebbf5b thru 0x3ec816b (0xc211 bytes) 2024-09-13T05:02:10.831 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:10.831+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:10.833 INFO:teuthology.orchestra.run.smithi053.stdout:80 trunc from 0xc3779d8 to 0xdeefaf0 2024-09-13T05:02:10.837 INFO:teuthology.orchestra.run.smithi053.stdout:81 read 0x18218b3 thru 0x1829011 (0x775f bytes) 2024-09-13T05:02:10.837 INFO:teuthology.orchestra.run.smithi053.stdout:82 write 0x41bdfc7 thru 0x41c2578 (0x45b2 bytes) 2024-09-13T05:02:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:83 read 0x13c56d5 thru 0x13c9d05 (0x4631 bytes) 2024-09-13T05:02:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:86 read 0x18e36ba thru 0x18f09f6 (0xd33d bytes) 2024-09-13T05:02:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:87 read 0x80b09a7 thru 0x80bde5d (0xd4b7 bytes) 2024-09-13T05:02:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:88 punch from 0x4044182 to 0x4053e10, (0xfc8e bytes) 2024-09-13T05:02:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.841 INFO:teuthology.orchestra.run.smithi053.stdout:90 write 0x6f0bfe thru 0x6f5a92 (0x4e95 bytes) 2024-09-13T05:02:10.843 INFO:teuthology.orchestra.run.smithi053.stdout:91 write 0xc73f23e thru 0xc743eef (0x4cb2 bytes) 2024-09-13T05:02:10.847 INFO:teuthology.orchestra.run.smithi053.stdout:92 writesame 0x4b5ca22 thru 0x4b69adf data_size 0x4d(0xd0be bytes) 2024-09-13T05:02:10.850 INFO:teuthology.orchestra.run.smithi053.stdout:93 write 0xd1e20b9 thru 0xd1ee8b6 (0xc7fe bytes) 2024-09-13T05:02:10.853 INFO:teuthology.orchestra.run.smithi053.stdout:94 write 0xc1ed1f0 thru 0xc1fbf49 (0xed5a bytes) 2024-09-13T05:02:10.855 INFO:teuthology.orchestra.run.smithi053.stdout:95 read 0xb652748 thru 0xb6587b7 (0x6070 bytes) 2024-09-13T05:02:10.855 INFO:teuthology.orchestra.run.smithi053.stdout:97 read 0x3195737 thru 0x31985e6 (0x2eb0 bytes) 2024-09-13T05:02:10.855 INFO:teuthology.orchestra.run.smithi053.stdout:98 read 0xfdaa0a thru 0xfe09c2 (0x5fb9 bytes) 2024-09-13T05:02:10.855 INFO:teuthology.orchestra.run.smithi053.stdout:99 write 0x4c296d2 thru 0x4c2c496 (0x2dc5 bytes) 2024-09-13T05:02:10.858 INFO:teuthology.orchestra.run.smithi053.stdout:100 write 0xd82dbd1 thru 0xd83cac7 (0xeef7 bytes) 2024-09-13T05:02:10.872 INFO:teuthology.orchestra.run.smithi053.stdout:101 trunc from 0xdeefaf0 to 0x4ce0475 2024-09-13T05:02:10.882 INFO:teuthology.orchestra.run.smithi053.stdout:103 punch from 0xf701de to 0xf7a605, (0xa427 bytes) 2024-09-13T05:02:10.889 INFO:teuthology.orchestra.run.smithi053.stdout:104 write 0x57fa4d7 thru 0x5804320 (0x9e4a bytes) 2024-09-13T05:02:10.898 INFO:teuthology.orchestra.run.smithi053.stdout:106 writesame 0x678c4c8 thru 0x679229d data_size 0x1(0x5dd6 bytes) 2024-09-13T05:02:10.899 INFO:teuthology.orchestra.run.smithi053.stdout:107 read 0x46d1eb6 thru 0x46dee40 (0xcf8b bytes) 2024-09-13T05:02:10.899 INFO:teuthology.orchestra.run.smithi053.stdout:108 punch from 0x39848f5 to 0x39920ec, (0xd7f7 bytes) 2024-09-13T05:02:10.899 INFO:teuthology.orchestra.run.smithi053.stdout:109 write 0x1ab66ea thru 0x1abd802 (0x7119 bytes) 2024-09-13T05:02:10.900 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:02:10.901 INFO:teuthology.orchestra.run.smithi053.stdout:111 read 0x8a5dc4 thru 0x8a75db (0x1818 bytes) 2024-09-13T05:02:10.911 INFO:teuthology.orchestra.run.smithi053.stdout:112 writesame 0x94a8764 thru 0x94a9779 data_size 0x1d(0x1016 bytes) 2024-09-13T05:02:10.911 INFO:teuthology.orchestra.run.smithi053.stdout:113 read 0x257ae2c thru 0x257e2e3 (0x34b8 bytes) 2024-09-13T05:02:10.911 INFO:teuthology.orchestra.run.smithi053.stdout:114 punch from 0x78a154a to 0x78af657, (0xe10d bytes) 2024-09-13T05:02:10.911 INFO:teuthology.orchestra.run.smithi053.stdout:115 read 0x19f9e08 thru 0x1a05619 (0xb812 bytes) 2024-09-13T05:02:10.912 INFO:teuthology.orchestra.run.smithi053.stdout:116 punch from 0x6f1b08d to 0x6f1bb8c, (0xaff bytes) 2024-09-13T05:02:10.912 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.915 INFO:teuthology.orchestra.run.smithi053.stdout:118 trunc from 0x94a977a to 0x570f563 2024-09-13T05:02:10.932 INFO:teuthology.orchestra.run.smithi053.stdout:119 trunc from 0x570f563 to 0x987d18 2024-09-13T05:02:10.932 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:10.931+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:10.942 INFO:teuthology.orchestra.run.smithi053.stdout:120 read 0x9565dd thru 0x95811e (0x1b42 bytes) 2024-09-13T05:02:10.942 INFO:teuthology.orchestra.run.smithi053.stdout:122 punch from 0x160002 to 0x1662f2, (0x62f0 bytes) 2024-09-13T05:02:10.942 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.942 INFO:teuthology.orchestra.run.smithi053.stdout:125 punch from 0x9626a8 to 0x969c9b, (0x75f3 bytes) 2024-09-13T05:02:10.942 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.954 INFO:teuthology.orchestra.run.smithi053.stdout:130 trunc from 0x987d18 to 0xbd0936a 2024-09-13T05:02:10.959 INFO:teuthology.orchestra.run.smithi053.stdout:131 writesame 0x7a29333 thru 0x7a32fb6 data_size 0x9f(0x9c84 bytes) 2024-09-13T05:02:10.963 INFO:teuthology.orchestra.run.smithi053.stdout:132 trunc from 0xbd0936a to 0xaa3fb12 2024-09-13T05:02:10.967 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:10.967 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:10.968 INFO:teuthology.orchestra.run.smithi053.stdout:133 write 0xf68ca4 thru 0xf6f59e (0x68fb bytes) 2024-09-13T05:02:10.971 INFO:teuthology.orchestra.run.smithi053.stdout:136 write 0x8b6298d thru 0x8b6e641 (0xbcb5 bytes) 2024-09-13T05:02:10.973 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.974 INFO:teuthology.orchestra.run.smithi053.stdout:139 read 0x27fc2c5 thru 0x28027c4 (0x6500 bytes) 2024-09-13T05:02:10.974 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.977 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:10.983 INFO:teuthology.orchestra.run.smithi053.stdout:141 writesame 0xeda67e0 thru 0xedb430a data_size 0x1(0xdb2b bytes) 2024-09-13T05:02:10.984 INFO:teuthology.orchestra.run.smithi053.stdout:146 read 0x6e540e5 thru 0x6e63c03 (0xfb1f bytes) 2024-09-13T05:02:10.984 INFO:teuthology.orchestra.run.smithi053.stdout:147 write 0xcda6c9e thru 0xcdb0460 (0x97c3 bytes) 2024-09-13T05:02:10.987 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:10.987 INFO:teuthology.orchestra.run.smithi053.stdout:149 writesame 0xbe0eccc thru 0xbe1c27a data_size 0x1(0xd5af bytes) 2024-09-13T05:02:11.013 INFO:teuthology.orchestra.run.smithi053.stdout:150 write 0xb3d4e7a thru 0xb3d7abe (0x2c45 bytes) 2024-09-13T05:02:11.021 INFO:teuthology.orchestra.run.smithi053.stdout:153 write 0xee140c2 thru 0xee1e5c5 (0xa504 bytes) 2024-09-13T05:02:11.024 INFO:teuthology.orchestra.run.smithi053.stdout:155 write 0xd3679ad thru 0xd36e12f (0x6783 bytes) 2024-09-13T05:02:11.027 INFO:teuthology.orchestra.run.smithi053.stdout:156 write 0x2948b90 thru 0x294e174 (0x55e5 bytes) 2024-09-13T05:02:11.030 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.032 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:11.032+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:11.033 INFO:teuthology.orchestra.run.smithi053.stdout:158 trunc from 0xee1e5c6 to 0xc6f8cfa 2024-09-13T05:02:11.042 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.045 INFO:teuthology.orchestra.run.smithi053.stdout:161 trunc from 0xc6f8cfa to 0x9d3e860 2024-09-13T05:02:11.055 INFO:teuthology.orchestra.run.smithi053.stdout:163 read 0x24c2f53 thru 0x24c6a0e (0x3abc bytes) 2024-09-13T05:02:11.058 INFO:teuthology.orchestra.run.smithi053.stdout:165 trunc from 0x9d3e860 to 0x769d172 2024-09-13T05:02:11.077 INFO:teuthology.orchestra.run.smithi053.stdout:166 writesame 0xcb0b8f4 thru 0xcb1a05a data_size 0x1(0xe767 bytes) 2024-09-13T05:02:11.082 INFO:teuthology.orchestra.run.smithi053.stdout:169 write 0xd1e6b7b thru 0xd1e8bd4 (0x205a bytes) 2024-09-13T05:02:11.085 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.086 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.086 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.089 INFO:teuthology.orchestra.run.smithi053.stdout:172 trunc from 0xd1e8bd5 to 0x9d068be 2024-09-13T05:02:11.095 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:11.098 INFO:teuthology.orchestra.run.smithi053.stdout:174 read 0x86b1586 thru 0x86b617f (0x4bfa bytes) 2024-09-13T05:02:11.105 INFO:teuthology.orchestra.run.smithi053.stdout:175 write 0xb433de9 thru 0xb43ade6 (0x6ffe bytes) 2024-09-13T05:02:11.108 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.108 INFO:teuthology.orchestra.run.smithi053.stdout:178 write 0x833da2a thru 0x8346609 (0x8be0 bytes) 2024-09-13T05:02:11.111 INFO:teuthology.orchestra.run.smithi053.stdout:179 punch from 0x2e917ab to 0x2ea0ef5, (0xf74a bytes) 2024-09-13T05:02:11.112 INFO:teuthology.orchestra.run.smithi053.stdout:180 write 0x46ddfb4 thru 0x46ed28d (0xf2da bytes) 2024-09-13T05:02:11.115 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.115 INFO:teuthology.orchestra.run.smithi053.stdout:185 write 0x48be63b thru 0x48c7c2b (0x95f1 bytes) 2024-09-13T05:02:11.118 INFO:teuthology.orchestra.run.smithi053.stdout:187 writesame 0x7e82859 thru 0x7e85a96 data_size 0x3b(0x323e bytes) 2024-09-13T05:02:11.121 INFO:teuthology.orchestra.run.smithi053.stdout:188 write 0x8169115 thru 0x816fb9c (0x6a88 bytes) 2024-09-13T05:02:11.122 INFO:teuthology.orchestra.run.smithi053.stdout:189 writesame 0x679c3bd thru 0x679f600 data_size 0x2(0x3244 bytes) 2024-09-13T05:02:11.126 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.126 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:11.126 INFO:teuthology.orchestra.run.smithi053.stdout:192 clone 1 order 23 su 262144 sc 11 2024-09-13T05:02:11.139 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:11.139+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:11.200 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.200 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.208 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:11.250 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:11.249+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:11.334 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:11.333+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:11.370 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.370 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.371 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:11.434 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:11.433+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:11.472 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.472 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.481 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:11.565 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:11.564+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:11.623 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.624 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.632 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:11.635 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:11.635+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:11.692 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:11.693 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:11.693 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T05:02:11.693 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T05:02:11.737 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:11.736+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:11.775 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.775 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.784 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:11.837 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:11.837+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:11.888 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:02:11.931 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:11.931 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:11.938 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:11.937+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:11.940 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:11.997 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 23, 2024-09-13T05:02:11.997 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "01e1476b-1467-4e0e-b01c-0f8cdf8276fe", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-13T05:01:45.075461+0000", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-13T05:02:11.885086+0000", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-13T05:01:55.160352+0000", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-13T05:01:47.179750+0000", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5804032, 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:11.998 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 4, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "reef", 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:11.999 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.817858+0000", 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:12.000 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:12.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "17", 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 3, 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 17, 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:12.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:12.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:12.004 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.981425+0000", 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:12.006 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:12.007 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "16", 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:12.008 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:12.009 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:12.010 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 8, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 8, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.25, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:02:12.011 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:02:05.508713+0000", 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:12.012 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-13T05:02:12.013 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "23", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:12.014 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 4, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 23, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:12.015 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:12.016 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.5, 2024-09-13T05:02:12.017 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.5, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.5, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.5, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:02:12.018 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "baf16a5c-4204-424b-8d8e-a5b1486aa052", 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.019 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.020 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.021 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.022 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1769628429", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1769628429", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1769628429", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1769628429", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.024 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f3df9a0f-3a9e-4665-9233-718bc0fe1223", 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.025 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.026 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-13T05:02:12.028 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-13T05:02:12.029 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/827277153", 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/827277153", 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/827277153", 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/827277153", 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.030 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1304a876-75c5-454b-a60b-8c9823c4aca8", 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 0, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.033 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.034 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/1334123504", 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/1334123504", 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/1334123504", 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/1334123504", 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "83668012-8ddb-4fe5-bb31-6e78d2723f12", 2024-09-13T05:02:12.036 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.037 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6824", 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6825", 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.038 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6826", 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6827", 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6830", 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6831", 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.040 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6828", 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6829", 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6825/4134389313", 2024-09-13T05:02:12.041 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6827/4134389313", 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6831/4134389313", 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6829/4134389313", 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f462232f-5144-4652-9139-9039d53e191a", 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.042 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6808", 2024-09-13T05:02:12.043 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6809", 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6810", 2024-09-13T05:02:12.044 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6811", 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.045 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6815", 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6816", 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.046 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6812", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6813", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6809/3681797816", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6811/3681797816", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6816/3681797816", 2024-09-13T05:02:12.047 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6813/3681797816", 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1d2d8607-f1d9-45e3-8486-3f5bb23b516b", 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.048 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6800", 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6801", 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.050 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6802", 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6803", 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6806", 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6807", 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.052 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6804", 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6805", 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.053 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6801/81482652", 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6803/81482652", 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6807/81482652", 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6805/81482652", 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "7fe0fb7d-0a40-4666-82a1-4e18f53b445c", 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.054 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6814", 2024-09-13T05:02:12.055 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6817", 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.056 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6818", 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6819", 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6822", 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6823", 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.058 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6820", 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6821", 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:12.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6817/576600432", 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6819/576600432", 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6823/576600432", 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6821/576600432", 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.060 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "811503be-abe3-4d64-a094-09ff675f3666", 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:12.061 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6824", 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6825", 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:12.062 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6826", 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6827", 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.063 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6830", 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6831", 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6828", 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6829", 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6825/896779674", 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6827/896779674", 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6831/896779674", 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6829/896779674", 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:02:12.066 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358655+0000", 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.067 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.343203+0000", 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.307081+0000", 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.315153+0000", 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:02:12.069 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.280902+0000", 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.070 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.302500+0000", 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.354685+0000", 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358223+0000", 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-13T05:02:12.072 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-13T05:02:12.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [ 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 2, 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 1 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-09-13T05:02:12.074 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.075 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:12.077 INFO:tasks.thrashosds.thrasher:j is {'epoch': 23, 'fsid': '01e1476b-1467-4e0e-b01c-0f8cdf8276fe', 'created': '2024-09-13T05:01:45.075461+0000', 'modified': '2024-09-13T05:02:11.885086+0000', 'last_up_change': '2024-09-13T05:01:55.160352+0000', 'last_in_change': '2024-09-13T05:01:47.179750+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T05:01:55.817858+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T05:01:55.981425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T05:02:05.508713+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 4, 'snap_epoch': 23, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'baf16a5c-4204-424b-8d8e-a5b1486aa052', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1769628429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1769628429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1769628429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1769628429}]}, 'public_addr': '172.21.15.53:6817/1769628429', 'cluster_addr': '172.21.15.53:6819/1769628429', 'heartbeat_back_addr': '172.21.15.53:6823/1769628429', 'heartbeat_front_addr': '172.21.15.53:6821/1769628429', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f3df9a0f-3a9e-4665-9233-718bc0fe1223', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 827277153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 827277153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 827277153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 827277153}]}, 'public_addr': '172.21.15.53:6801/827277153', 'cluster_addr': '172.21.15.53:6803/827277153', 'heartbeat_back_addr': '172.21.15.53:6807/827277153', 'heartbeat_front_addr': '172.21.15.53:6805/827277153', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1304a876-75c5-454b-a60b-8c9823c4aca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 1334123504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 1334123504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 1334123504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 1334123504}]}, 'public_addr': '172.21.15.53:6809/1334123504', 'cluster_addr': '172.21.15.53:6811/1334123504', 'heartbeat_back_addr': '172.21.15.53:6815/1334123504', 'heartbeat_front_addr': '172.21.15.53:6813/1334123504', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '83668012-8ddb-4fe5-bb31-6e78d2723f12', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6824', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6825', 'nonce': 4134389313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6826', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6827', 'nonce': 4134389313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6830', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6831', 'nonce': 4134389313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6828', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6829', 'nonce': 4134389313}]}, 'public_addr': '172.21.15.53:6825/4134389313', 'cluster_addr': '172.21.15.53:6827/4134389313', 'heartbeat_back_addr': '172.21.15.53:6831/4134389313', 'heartbeat_front_addr': '172.21.15.53:6829/4134389313', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f462232f-5144-4652-9139-9039d53e191a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 3681797816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 3681797816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6815', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6816', 'nonce': 3681797816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 3681797816}]}, 'public_addr': '172.21.15.110:6809/3681797816', 'cluster_addr': '172.21.15.110:6811/3681797816', 'heartbeat_back_addr': '172.21.15.110:6816/3681797816', 'heartbeat_front_addr': '172.21.15.110:6813/3681797816', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1d2d8607-f1d9-45e3-8486-3f5bb23b516b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 81482652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 81482652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 81482652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 81482652}]}, 'public_addr': '172.21.15.110:6801/81482652', 'cluster_addr': '172.21.15.110:6803/81482652', 'heartbeat_back_addr': '172.21.15.110:6807/81482652', 'heartbeat_front_addr': '172.21.15.110:6805/81482652', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7fe0fb7d-0a40-4666-82a1-4e18f53b445c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 576600432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 576600432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 576600432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 576600432}]}, 'public_addr': '172.21.15.110:6817/576600432', 'cluster_addr': '172.21.15.110:6819/576600432', 'heartbeat_back_addr': '172.21.15.110:6823/576600432', 'heartbeat_front_addr': '172.21.15.110:6821/576600432', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '811503be-abe3-4d64-a094-09ff675f3666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6824', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6825', 'nonce': 896779674}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6826', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6827', 'nonce': 896779674}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6830', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6831', 'nonce': 896779674}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6828', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6829', 'nonce': 896779674}]}, 'public_addr': '172.21.15.110:6825/896779674', 'cluster_addr': '172.21.15.110:6827/896779674', 'heartbeat_back_addr': '172.21.15.110:6831/896779674', 'heartbeat_front_addr': '172.21.15.110:6829/896779674', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358655+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.343203+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.307081+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.315153+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.280902+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.302500+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.354685+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T05:02:12.077 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-13T05:02:12.077 INFO:teuthology.orchestra.run.smithi053.stdout:193 trunc from 0xb43ade7 to 0x289aaa1 2024-09-13T05:02:12.078 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:12.076+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:12.081 INFO:teuthology.orchestra.run.smithi053.stdout:195 trunc from 0x289aaa1 to 0x5f0f13e 2024-09-13T05:02:12.085 INFO:teuthology.orchestra.run.smithi053.stdout:196 read 0x371ffd1 thru 0x37209bf (0x9ef bytes) 2024-09-13T05:02:12.086 INFO:teuthology.orchestra.run.smithi053.stdout:197 read 0x1fd4191 thru 0x1fdf205 (0xb075 bytes) 2024-09-13T05:02:12.090 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:12.091 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:12.100 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:12.100 INFO:teuthology.orchestra.run.smithi053.stdout:198 writesame 0xdd4125f thru 0xdd4ebe5 data_size 0x1(0xd987 bytes) 2024-09-13T05:02:12.101 INFO:teuthology.orchestra.run.smithi053.stdout:199 write 0x628c56b thru 0x62948ad (0x8343 bytes) 2024-09-13T05:02:12.104 INFO:teuthology.orchestra.run.smithi053.stdout:200 read 0x63c97f2 thru 0x63ca3e3 (0xbf2 bytes) 2024-09-13T05:02:12.104 INFO:teuthology.orchestra.run.smithi053.stdout:203 write 0x2f37e61 thru 0x2f39685 (0x1825 bytes) 2024-09-13T05:02:12.105 INFO:teuthology.orchestra.run.smithi053.stdout:205 write 0x41dcab3 thru 0x41e85ce (0xbb1c bytes) 2024-09-13T05:02:12.107 INFO:teuthology.orchestra.run.smithi053.stdout:206 writesame 0x75e2b9b thru 0x75eb246 data_size 0x22(0x86ac bytes) 2024-09-13T05:02:12.108 INFO:teuthology.orchestra.run.smithi053.stdout:207 writesame 0x1fe76b4 thru 0x1fee4b4 data_size 0x1b(0x6e01 bytes) 2024-09-13T05:02:12.115 INFO:teuthology.orchestra.run.smithi053.stdout:208 writesame 0xe41111b thru 0xe417bcd data_size 0xf(0x6ab3 bytes) 2024-09-13T05:02:12.115 INFO:teuthology.orchestra.run.smithi053.stdout:209 writesame 0x609bd0e thru 0x609f3fd data_size 0x8(0x36f0 bytes) 2024-09-13T05:02:12.118 INFO:teuthology.orchestra.run.smithi053.stdout:210 read 0x5c4e829 thru 0x5c51be3 (0x33bb bytes) 2024-09-13T05:02:12.118 INFO:teuthology.orchestra.run.smithi053.stdout:213 read 0x7f1beae thru 0x7f29257 (0xd3aa bytes) 2024-09-13T05:02:12.119 INFO:teuthology.orchestra.run.smithi053.stdout:214 write 0x8a7152a thru 0x8a7d046 (0xbb1d bytes) 2024-09-13T05:02:12.122 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.122 INFO:teuthology.orchestra.run.smithi053.stdout:216 punch from 0xbc41d0d to 0xbc42151, (0x444 bytes) 2024-09-13T05:02:12.122 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.122 INFO:teuthology.orchestra.run.smithi053.stdout:220 writesame 0xa567ffd thru 0xa5762dd data_size 0x1(0xe2e1 bytes) 2024-09-13T05:02:12.149 INFO:teuthology.orchestra.run.smithi053.stdout:222 write 0xc6fa3d3 thru 0xc7093d1 (0xefff bytes) 2024-09-13T05:02:12.153 INFO:teuthology.orchestra.run.smithi053.stdout:223 read 0x7f80e9a thru 0x7f8f0f1 (0xe258 bytes) 2024-09-13T05:02:12.153 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.153 INFO:teuthology.orchestra.run.smithi053.stdout:225 writesame 0x2bdc1f9 thru 0x2be8fcd data_size 0x1d(0xcdd5 bytes) 2024-09-13T05:02:12.154 INFO:teuthology.orchestra.run.smithi053.stdout:227 read 0x9131492 thru 0x913ba3d (0xa5ac bytes) 2024-09-13T05:02:12.155 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.155 INFO:teuthology.orchestra.run.smithi053.stdout:229 read 0x6b00cea thru 0x6b0c96c (0xbc83 bytes) 2024-09-13T05:02:12.155 INFO:teuthology.orchestra.run.smithi053.stdout:231 write 0x55b26c7 thru 0x55b28de (0x218 bytes) 2024-09-13T05:02:12.156 INFO:teuthology.orchestra.run.smithi053.stdout:232 writesame 0x289f9f1 thru 0x28ac116 data_size 0x3(0xc726 bytes) 2024-09-13T05:02:12.162 INFO:teuthology.orchestra.run.smithi053.stdout:234 read 0xaca3d8 thru 0xad6d47 (0xc970 bytes) 2024-09-13T05:02:12.169 INFO:teuthology.orchestra.run.smithi053.stdout:236 trunc from 0xe417bce to 0x82f8675 2024-09-13T05:02:12.177 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:12.177+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:12.180 INFO:teuthology.orchestra.run.smithi053.stdout:237 punch from 0x53fc44b to 0x54096ff, (0xd2b4 bytes) 2024-09-13T05:02:12.182 INFO:teuthology.orchestra.run.smithi053.stdout:239 trunc from 0x82f8675 to 0x5c86318 2024-09-13T05:02:12.189 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.189 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.190 INFO:teuthology.orchestra.run.smithi053.stdout:243 trunc from 0x5c86318 to 0x568f39b 2024-09-13T05:02:12.208 INFO:teuthology.orchestra.run.smithi053.stdout:244 write 0x8824a93 thru 0x88274b8 (0x2a26 bytes) 2024-09-13T05:02:12.211 INFO:teuthology.orchestra.run.smithi053.stdout:245 trunc from 0x88274b9 to 0x844e5d9 2024-09-13T05:02:12.223 INFO:teuthology.orchestra.run.smithi053.stdout:246 trunc from 0x844e5d9 to 0x3b9522f 2024-09-13T05:02:12.234 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:12.249 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:12.250 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:12.252 INFO:teuthology.orchestra.run.smithi053.stdout:249 write 0xe27e7b2 thru 0xe28e1fa (0xfa49 bytes) 2024-09-13T05:02:12.255 INFO:teuthology.orchestra.run.smithi053.stdout:250 punch from 0xc4d67ea to 0xc4e18b8, (0xb0ce bytes) 2024-09-13T05:02:12.255 INFO:teuthology.orchestra.run.smithi053.stdout:251 read 0x8ed7db8 thru 0x8edd36f (0x55b8 bytes) 2024-09-13T05:02:12.255 INFO:teuthology.orchestra.run.smithi053.stdout:253 clone 2 order 23 su 262144 sc 6 2024-09-13T05:02:12.259 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:12.303 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:12.302+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:12.358 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:12.361 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:12.362 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:12.370 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:12.412 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:12.411+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:12.480 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:12.479+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:12.483 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:12.484 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:12.500 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:12.500 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:12.511 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:12.511 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:12.511 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:12.511 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:12.518 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:12.518 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:12.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:12.535 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:12.535 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:12.557 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:12.558 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:12.581 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:12.580+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:12.589 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:12.590 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:12.606 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:12.607 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:12.624 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:12.625 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:12.643 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:12.643 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:12.644 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:12.653 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:12.682 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:12.682+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:12.783 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:12.783+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:12.800 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4235.0:5 1.2 e16/14 1:5cea7035:::rbd_info:head v 16'1, mlcod=0'0)", 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.511925771, 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084071500000000002, 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287667+0000", 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287676+0000", 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6070000000000008e-06 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287692+0000", 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6341000000000002e-05 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287705+0000", 2024-09-13T05:02:12.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2951e-05 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287772+0000", 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7366000000000005e-05 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.287804+0000", 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1602e-05 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:12.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.288474+0000", 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067043599999999995 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:58.288508+0000", 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3657e-05 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4235.0:7 1.2 e17/14 1:5cea7035:::rbd_info:head v 17'2, mlcod=0'0)", 2024-09-13T05:02:12.805 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.602873795000001, 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091731799999999997, 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:12.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196719+0000", 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196726+0000", 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1980000000000004e-06 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196736+0000", 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0231000000000001e-05 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196744+0000", 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0229999999999992e-06 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196803+0000", 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9004e-05 2024-09-13T05:02:12.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.196833+0000", 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.94e-05 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.197605+0000", 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077226900000000002 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:59.197636+0000", 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1417000000000001e-05 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.888870+0000", 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9107234470000001, 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003860437, 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 11 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888870+0000", 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888870+0000", 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888867+0000", 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888880+0000", 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2403e-05 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888903+0000", 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3133e-05 2024-09-13T05:02:12.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888916+0000", 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2989e-05 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888967+0000", 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1168000000000002e-05 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:12.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.888972+0000", 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5809999999999998e-06 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.891272+0000", 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022999119999999999 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.891409+0000", 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000137491 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.891619+0000", 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021031000000000001 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892622+0000", 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010026919999999999 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892641+0000", 2024-09-13T05:02:12.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9045000000000001e-05 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892691+0000", 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-05 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892730+0000", 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9223e-05 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:11 3.1f e22/19 ondisk, result = 0)", 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.892522+0000", 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.907070719, 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105386, 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892522+0000", 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892522+0000", 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892521+0000", 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892527+0000", 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892538+0000", 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.028e-05 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892547+0000", 2024-09-13T05:02:12.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1929999999999999e-06 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892606+0000", 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8746999999999998e-05 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892614+0000", 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5599999999999994e-06 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892628+0000", 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3485999999999999e-05 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:12 3.1 3:86cf5969:::rbd_header.117c773772bd:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.892891+0000", 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9067017320000001, 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011613470000000001, 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:12.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 12 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892891+0000", 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892891+0000", 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892890+0000", 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892896+0000", 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9800000000000003e-06 2024-09-13T05:02:12.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892906+0000", 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9920000000000002e-06 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892911+0000", 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.916e-06 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.892962+0000", 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0921000000000003e-05 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893101+0000", 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000138651 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893337+0000", 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023641999999999999 2024-09-13T05:02:12.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893952+0000", 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061527900000000004 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893963+0000", 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0411000000000001e-05 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:12.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.894010+0000", 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7679000000000002e-05 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.894053+0000", 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2191000000000003e-05 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:12 3.1 e22/19 ondisk, result = 0)", 2024-09-13T05:02:12.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.893888+0000", 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9057051500000002, 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7738999999999996e-05, 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893888+0000", 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893888+0000", 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893887+0000", 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893891+0000", 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1969999999999998e-06 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893901+0000", 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5030000000000003e-06 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893905+0000", 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.301e-06 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893940+0000", 2024-09-13T05:02:12.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4476e-05 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893947+0000", 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3690000000000001e-06 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.893956+0000", 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6649999999999992e-06 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:14 3.1f 3:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.898546+0000", 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.90104739, 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028609399999999998, 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.832 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 14 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898546+0000", 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898546+0000", 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898543+0000", 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898560+0000", 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7146000000000002e-05 2024-09-13T05:02:12.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898584+0000", 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4338999999999999e-05 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898596+0000", 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1292e-05 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898658+0000", 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2566999999999999e-05 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898755+0000", 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7054999999999998e-05 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898832+0000", 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6481000000000002e-05 2024-09-13T05:02:12.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:15 3.1f 3:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.898943+0000", 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9006499990000001, 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029036300000000003, 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 15 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898943+0000", 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898943+0000", 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898942+0000", 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898948+0000", 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3899999999999998e-06 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898961+0000", 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2622e-05 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.898969+0000", 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899012+0000", 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3007999999999999e-05 2024-09-13T05:02:12.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899062+0000", 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9966000000000002e-05 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899233+0000", 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017133 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:16 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.899346+0000", 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.900246793, 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000235956, 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 16 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899346+0000", 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899346+0000", 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899345+0000", 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899352+0000", 2024-09-13T05:02:12.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3470000000000004e-06 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899362+0000", 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7659999999999993e-06 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899367+0000", 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1449999999999999e-06 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899400+0000", 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3033e-05 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899446+0000", 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6174999999999998e-05 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899582+0000", 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013624199999999999 2024-09-13T05:02:12.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:17 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8999256620000002, 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015649500000000001, 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 17 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899667+0000", 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899671+0000", 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6959999999999996e-06 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899680+0000", 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2460000000000003e-06 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899684+0000", 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.941e-06 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899710+0000", 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6392000000000001e-05 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899750+0000", 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0522999999999998e-05 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899824+0000", 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3324999999999993e-05 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:18 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8996936720000002, 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016291299999999999, 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 18 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899899+0000", 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899903+0000", 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1930000000000003e-06 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899911+0000", 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8290000000000008e-06 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899915+0000", 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9199999999999997e-06 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899940+0000", 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5398999999999999e-05 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.899983+0000", 2024-09-13T05:02:12.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2531000000000003e-05 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900062+0000", 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9677000000000003e-05 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:19 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.856 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.900149+0000", 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.899444028, 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014279900000000001, 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 19 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900149+0000", 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900149+0000", 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900148+0000", 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900153+0000", 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9389999999999996e-06 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900162+0000", 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5310000000000001e-06 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900166+0000", 2024-09-13T05:02:12.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0400000000000003e-06 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900191+0000", 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5307e-05 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900230+0000", 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8770999999999999e-05 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900292+0000", 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1852000000000003e-05 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:20 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.900645+0000", 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.89894841, 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00032938800000000001, 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 20 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900645+0000", 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900645+0000", 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900644+0000", 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900648+0000", 2024-09-13T05:02:12.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.515e-06 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900658+0000", 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1600000000000004e-06 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900662+0000", 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900689+0000", 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6472e-05 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900734+0000", 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5352000000000002e-05 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.900974+0000", 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000239812 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:21 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.901066+0000", 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8985274539999999, 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000233494, 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 21 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901066+0000", 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901066+0000", 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901065+0000", 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901070+0000", 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3720000000000004e-06 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901079+0000", 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2629999999999992e-06 2024-09-13T05:02:12.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901084+0000", 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6929999999999998e-06 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901117+0000", 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2628e-05 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901161+0000", 2024-09-13T05:02:12.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4267999999999997e-05 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901299+0000", 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013798099999999999 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:22 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8982135100000002, 2024-09-13T05:02:12.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014782700000000001, 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 22 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901379+0000", 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901383+0000", 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.794e-06 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901390+0000", 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.807e-06 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901394+0000", 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8010000000000001e-06 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901419+0000", 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5159000000000001e-05 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901457+0000", 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7589000000000002e-05 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.901527+0000", 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0290999999999995e-05 2024-09-13T05:02:12.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:24 3.1 3:86cf5969:::rbd_header.117c773772bd:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.902237+0000", 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.897355846, 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016992799999999999, 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 24 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902237+0000", 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902237+0000", 2024-09-13T05:02:12.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902236+0000", 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902241+0000", 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2729999999999996e-06 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902250+0000", 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9390000000000003e-06 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902254+0000", 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1289999999999998e-06 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902289+0000", 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5317000000000001e-05 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902337+0000", 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8010000000000003e-05 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902407+0000", 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9931999999999994e-05 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.879 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:25 3.1 3:86cf5969:::rbd_header.117c773772bd:head [watch watch cookie 140166391944464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.902537+0000", 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.897056026, 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000975175, 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 25 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902537+0000", 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902537+0000", 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902536+0000", 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:12.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902541+0000", 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.566e-06 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902550+0000", 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7350000000000002e-06 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902560+0000", 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.01e-05 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902591+0000", 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1683000000000001e-05 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902638+0000", 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6592999999999999e-05 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.902725+0000", 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7255000000000004e-05 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903265+0000", 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054003199999999997 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903435+0000", 2024-09-13T05:02:12.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016971099999999999 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903460+0000", 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5174e-05 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903512+0000", 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1952e-05 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:25 3.1 e22/19 ondisk, result = 0)", 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:08.903385+0000", 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8962080210000001, 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013565800000000001, 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:12.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903385+0000", 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903385+0000", 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903384+0000", 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903391+0000", 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9829999999999999e-06 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903400+0000", 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4950000000000008e-06 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903405+0000", 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0629999999999996e-06 2024-09-13T05:02:12.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903427+0000", 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2419999999999999e-05 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903432+0000", 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7679999999999999e-06 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:08.903521+0000", 2024-09-13T05:02:12.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8727000000000006e-05 2024-09-13T05:02:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:12.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:12.890 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:12.891 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:12.890+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:12.892 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:12.952 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone1 intact 2024-09-13T05:02:12.992 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:12.991+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:13.022 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:13.022 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:13.022 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:13.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:13.093 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:13.093+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:13.111 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-13T05:02:13.148 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:13.148 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:13.148 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:13.160 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:13.194 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:13.193+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4235.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.179043+0000", 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.094122884000001, 2024-09-13T05:02:13.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001385266, 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4235", 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/2115106039", 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179043+0000", 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179043+0000", 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179041+0000", 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179052+0000", 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1083e-05 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179067+0000", 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4440000000000001e-05 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179082+0000", 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5537e-05 2024-09-13T05:02:13.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179165+0000", 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2551000000000003e-05 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179384+0000", 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021954400000000001 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6", 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.179542+0000", 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015789999999999999 2024-09-13T05:02:13.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180067+0000", 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052489400000000003 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180347+0000", 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027953800000000002 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180379+0000", 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2577999999999999e-05 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.180428+0000", 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9308999999999998e-05 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:20 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'10, mlcod=15'8)", 2024-09-13T05:02:13.281 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.195103+0000", 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.078062891999998, 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001412188, 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195103+0000", 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195103+0000", 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195102+0000", 2024-09-13T05:02:13.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195214+0000", 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000111531 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195227+0000", 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3400999999999999e-05 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195233+0000", 2024-09-13T05:02:13.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3480000000000003e-06 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195265+0000", 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1983000000000002e-05 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195281+0000", 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6218e-05 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196474+0000", 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001193127 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196515+0000", 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1440000000000003e-05 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:21 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'11, mlcod=15'8)", 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.195632+0000", 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.077534441000001, 2024-09-13T05:02:13.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098342500000000001, 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195632+0000", 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195632+0000", 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195631+0000", 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195693+0000", 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2576999999999994e-05 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195706+0000", 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3178e-05 2024-09-13T05:02:13.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195713+0000", 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2589999999999997e-06 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195752+0000", 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9752999999999997e-05 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.195777+0000", 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4584999999999999e-05 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196582+0000", 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080502500000000001 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.196615+0000", 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3161999999999999e-05 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.290 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:31 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'17, mlcod=15'14)", 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.202696+0000", 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.070470066999999, 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010801249999999999, 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202696+0000", 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202696+0000", 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202695+0000", 2024-09-13T05:02:13.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202712+0000", 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6399999999999999e-05 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202720+0000", 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.909e-06 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202724+0000", 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8480000000000003e-06 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202763+0000", 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9631000000000001e-05 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.202788+0000", 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5239e-05 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203741+0000", 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095256099999999999 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203776+0000", 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5221e-05 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.295 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:32 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'18, mlcod=15'14)", 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.203142+0000", 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.070024538999998, 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001006875, 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203142+0000", 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.296 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203142+0000", 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203141+0000", 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203187+0000", 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6737000000000003e-05 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203198+0000", 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0104e-05 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203202+0000", 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2749999999999997e-06 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203273+0000", 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0753999999999994e-05 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.203296+0000", 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2983999999999999e-05 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204112+0000", 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00081669499999999999 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204148+0000", 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6124000000000002e-05 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:35 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'17)", 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.204847+0000", 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.068319146, 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010255100000000001, 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.300 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204847+0000", 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204847+0000", 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204846+0000", 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204918+0000", 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1952000000000005e-05 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204929+0000", 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0489000000000001e-05 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204933+0000", 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3610000000000003e-06 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204971+0000", 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8037000000000002e-05 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.204991+0000", 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0259e-05 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.205830+0000", 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083891599999999999 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.205872+0000", 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2073000000000002e-05 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:38 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'24, mlcod=15'22)", 2024-09-13T05:02:13.304 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.207295+0000", 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.065871397999999, 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010032730000000001, 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207295+0000", 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207295+0000", 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207294+0000", 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207343+0000", 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8736999999999998e-05 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207354+0000", 2024-09-13T05:02:13.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1741e-05 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207359+0000", 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7979999999999996e-06 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207398+0000", 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9048999999999998e-05 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.207420+0000", 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1886999999999999e-05 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.208265+0000", 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084525099999999999 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.208298+0000", 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2564999999999997e-05 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:54 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'32, mlcod=15'29)", 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.218237+0000", 2024-09-13T05:02:13.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.054929021, 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010037360000000001, 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218237+0000", 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218237+0000", 2024-09-13T05:02:13.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218236+0000", 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218242+0000", 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2859999999999999e-06 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218250+0000", 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3380000000000005e-06 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218264+0000", 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3681e-05 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218302+0000", 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8099999999999998e-05 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.218325+0000", 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3263000000000001e-05 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.219205+0000", 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088005600000000005 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.219241+0000", 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5632e-05 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:57 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'35, mlcod=15'33)", 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.220620+0000", 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.052545780999999, 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001174504, 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220620+0000", 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220620+0000", 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220613+0000", 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999924 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220677+0000", 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4021000000000002e-05 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220688+0000", 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1399e-05 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220693+0000", 2024-09-13T05:02:13.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7129999999999996e-06 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220727+0000", 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4649e-05 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.220748+0000", 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0452e-05 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.221747+0000", 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099969899999999994 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.221795+0000", 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7345999999999997e-05 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:68 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'42, mlcod=15'39)", 2024-09-13T05:02:13.318 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.227836+0000", 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.045330147000001, 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099165400000000002, 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227836+0000", 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227836+0000", 2024-09-13T05:02:13.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227835+0000", 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227842+0000", 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4389999999999997e-06 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227855+0000", 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3162e-05 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227861+0000", 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2960000000000004e-06 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227891+0000", 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0266000000000001e-05 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.227911+0000", 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9704e-05 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228769+0000", 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085764800000000003 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.228828+0000", 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8909000000000001e-05 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:71 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'45, mlcod=15'43)", 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.230089+0000", 2024-09-13T05:02:13.323 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.043077414999999, 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010190189999999999, 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230089+0000", 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230089+0000", 2024-09-13T05:02:13.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230088+0000", 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230146+0000", 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8363999999999999e-05 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230158+0000", 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1403000000000001e-05 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230163+0000", 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230206+0000", 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3541999999999997e-05 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230228+0000", 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1994999999999999e-05 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231050+0000", 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00082189800000000005 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231108+0000", 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7670000000000002e-05 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:72 2.0 e15/14 2:30306672:devicehealth::main.db.0000000000000000:head v 15'46, mlcod=15'43)", 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.230536+0000", 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.042630135, 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001235676, 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230536+0000", 2024-09-13T05:02:13.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230536+0000", 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230535+0000", 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230607+0000", 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1557999999999995e-05 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230613+0000", 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.353e-06 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230617+0000", 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5530000000000001e-06 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230661+0000", 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3667000000000001e-05 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.230679+0000", 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8603000000000001e-05 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231743+0000", 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001063949 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.231772+0000", 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8461000000000001e-05 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:87 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'57, mlcod=15'52)", 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 16.033928757999998, 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00107336, 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239237+0000", 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239297+0000", 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9759999999999997e-05 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239307+0000", 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0631e-05 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239314+0000", 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5799999999999997e-06 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239347+0000", 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2789000000000001e-05 2024-09-13T05:02:13.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.239359+0000", 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2469999999999999e-05 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.240274+0000", 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091490799999999998 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.240311+0000", 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6806e-05 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4213.0:105 2.0 e15/14 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'67, mlcod=15'64)", 2024-09-13T05:02:13.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.892791679, 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010332659999999999, 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:13.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380374+0000", 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380379+0000", 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6380000000000003e-06 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380388+0000", 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3319999999999998e-06 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380392+0000", 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7139999999999999e-06 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380426+0000", 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4817000000000002e-05 2024-09-13T05:02:13.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.380451+0000", 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4533e-05 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.381384+0000", 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093344999999999995 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:01:57.381408+0000", 2024-09-13T05:02:13.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3302000000000002e-05 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:882 3.4 3:229424df:::rbd_header.117ca06e531b:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.896288+0000", 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.37687842799999999, 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003102254, 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 882 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896288+0000", 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896288+0000", 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896285+0000", 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896298+0000", 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3315e-05 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896322+0000", 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4025000000000001e-05 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896333+0000", 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0891e-05 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896381+0000", 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.812e-05 2024-09-13T05:02:13.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.896386+0000", 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6700000000000002e-06 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898069+0000", 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016835909999999999 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898143+0000", 2024-09-13T05:02:13.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4059000000000004e-05 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898378+0000", 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023432600000000001 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898958+0000", 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058014700000000004 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.899339+0000", 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038066199999999998 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.899367+0000", 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8578e-05 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.899390+0000", 2024-09-13T05:02:13.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2713000000000001e-05 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:896 3.4 3:229424df:::rbd_header.117ca06e531b:head [call rbd.set_protection_status in=24b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.908649+0000", 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.36451690799999997, 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098900500000000005, 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.353 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 896 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908649+0000", 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908649+0000", 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908649+0000", 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908652+0000", 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1949999999999999e-06 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908659+0000", 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908662+0000", 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7000000000000002e-06 2024-09-13T05:02:13.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908701+0000", 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8478e-05 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908749+0000", 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8508999999999999e-05 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908893+0000", 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000143153 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909389+0000", 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049600800000000004 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909588+0000", 2024-09-13T05:02:13.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019973500000000001 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909617+0000", 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8223999999999999e-05 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909638+0000", 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1676e-05 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.359 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:930 3.0 3:04479396:::rbd_id.image_client.0-clone2:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.930622+0000", 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.34254453800000001, 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015752369999999999, 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 930 2024-09-13T05:02:13.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930622+0000", 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930622+0000", 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930620+0000", 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930628+0000", 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8689999999999993e-06 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930644+0000", 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5678000000000001e-05 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930651+0000", 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6889999999999998e-06 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930712+0000", 2024-09-13T05:02:13.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0822999999999997e-05 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.930876+0000", 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016394599999999999 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931118+0000", 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000242193 2024-09-13T05:02:13.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931837+0000", 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071950099999999999 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.932144+0000", 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00030690399999999999 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.932174+0000", 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9286e-05 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.932197+0000", 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3285000000000002e-05 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:965 3.4 3:229424df:::rbd_header.117ca06e531b:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.080700+0000", 2024-09-13T05:02:13.366 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.19246571700000001, 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001145765, 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 965 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080700+0000", 2024-09-13T05:02:13.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080700+0000", 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080698+0000", 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080711+0000", 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2795e-05 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080733+0000", 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2368e-05 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080743+0000", 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6360000000000006e-06 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080803+0000", 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0309000000000001e-05 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080874+0000", 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1437000000000001e-05 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.080999+0000", 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000124082 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081621+0000", 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00062281499999999998 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081793+0000", 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017206799999999999 2024-09-13T05:02:13.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081822+0000", 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8371999999999999e-05 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081846+0000", 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4328000000000002e-05 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:967 3.4 3:229424df:::rbd_header.117ca06e531b:head [watch unwatch cookie 140166325266464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.081994+0000", 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.191171653, 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010025329999999999, 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 967 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081994+0000", 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081994+0000", 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081994+0000", 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081998+0000", 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-06 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082002+0000", 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0069999999999999e-06 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082004+0000", 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8770000000000002e-06 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082145+0000", 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014021699999999999 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082187+0000", 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2391999999999997e-05 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082274+0000", 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6629e-05 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082641+0000", 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036736100000000002 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082909+0000", 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026791100000000002 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082938+0000", 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8663e-05 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082997+0000", 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9370999999999998e-05 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:968 3.4 3:229424df:::rbd_header.117ca06e531b:head [notify-ack] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.082199+0000", 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.190966939, 2024-09-13T05:02:13.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099273900000000008, 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 968 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082199+0000", 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082199+0000", 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082198+0000", 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082203+0000", 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0520000000000001e-06 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082214+0000", 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0441e-05 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082221+0000", 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082441+0000", 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000220702 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for rw locks", 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082481+0000", 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9619999999999997e-05 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.083036+0000", 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055491700000000004 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.083111+0000", 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4824000000000001e-05 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.083192+0000", 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1188999999999998e-05 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.383 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:13.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:13.387 INFO:teuthology.orchestra.run.smithi053.stdout:254 write 0x15acd2b thru 0x15aeca6 (0x1f7c bytes) 2024-09-13T05:02:13.387 INFO:teuthology.orchestra.run.smithi053.stdout:255 write 0xaa62dea thru 0xaa6f1f6 (0xc40d bytes) 2024-09-13T05:02:13.387 INFO:teuthology.orchestra.run.smithi053.stdout:256 read 0x438bb22 thru 0x4398698 (0xcb77 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:258 read 0x1ca4e19 thru 0x1ca9fb1 (0x5199 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:260 write 0xa75d9ce thru 0xa76cd1a (0xf34d bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:261 write 0xb5ee4c0 thru 0xb5f43df (0x5f20 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:263 write 0xa53fb49 thru 0xa5428fe (0x2db6 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:264 read 0xb5957e0 thru 0xb59b233 (0x5a54 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:265 write 0x72bf3c8 thru 0x72ce580 (0xf1b9 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:268 read 0x769c8c1 thru 0x76ac169 (0xf8a9 bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:269 read 0xa7d303 thru 0xa8947f (0xc17d bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:271 trunc from 0xe28e1fb to 0xbc19317 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:275 write 0x254f72b thru 0x255b868 (0xc13e bytes) 2024-09-13T05:02:13.388 INFO:teuthology.orchestra.run.smithi053.stdout:277 trunc from 0xbc19317 to 0xb0cf971 2024-09-13T05:02:13.389 INFO:teuthology.orchestra.run.smithi053.stdout:278 trunc from 0xb0cf971 to 0xbafde22 2024-09-13T05:02:13.389 INFO:teuthology.orchestra.run.smithi053.stdout:279 writesame 0xa1e146f thru 0xa1e9bb6 data_size 0xc(0x8748 bytes) 2024-09-13T05:02:13.389 INFO:teuthology.orchestra.run.smithi053.stdout:280 punch from 0x7ee6c28 to 0x7ef1a9c, (0xae74 bytes) 2024-09-13T05:02:13.389 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:13.389 INFO:teuthology.orchestra.run.smithi053.stdout:282 trunc from 0xbafde22 to 0x18ad3bc 2024-09-13T05:02:13.389 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:13.384+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:13.392 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:13.392 INFO:teuthology.orchestra.run.smithi053.stdout:285 read 0x12d0d17 thru 0x12d3c18 (0x2f02 bytes) 2024-09-13T05:02:13.392 INFO:teuthology.orchestra.run.smithi053.stdout:286 read 0x104788b thru 0x1048b3b (0x12b1 bytes) 2024-09-13T05:02:13.394 INFO:teuthology.orchestra.run.smithi053.stdout:288 punch from 0x4641ed to 0x4663b7, (0x21ca bytes) 2024-09-13T05:02:13.406 INFO:teuthology.orchestra.run.smithi053.stdout:289 write 0x7f64bc0 thru 0x7f6e99f (0x9de0 bytes) 2024-09-13T05:02:13.410 INFO:teuthology.orchestra.run.smithi053.stdout:290 trunc from 0x7f6e9a0 to 0x80a9b0a 2024-09-13T05:02:13.418 INFO:teuthology.orchestra.run.smithi053.stdout:291 trunc from 0x80a9b0a to 0x51f6a82 2024-09-13T05:02:13.434 INFO:teuthology.orchestra.run.smithi053.stdout:292 write 0x789c1a3 thru 0x78a8293 (0xc0f1 bytes) 2024-09-13T05:02:13.437 INFO:teuthology.orchestra.run.smithi053.stdout:294 write 0x1a94031 thru 0x1a9b079 (0x7049 bytes) 2024-09-13T05:02:13.447 INFO:teuthology.orchestra.run.smithi053.stdout:295 write 0xa98744b thru 0xa994524 (0xd0da bytes) 2024-09-13T05:02:13.451 INFO:teuthology.orchestra.run.smithi053.stdout:298 read 0x6bbaa9c thru 0x6bbd54b (0x2ab0 bytes) 2024-09-13T05:02:13.451 INFO:teuthology.orchestra.run.smithi053.stdout:299 read 0x12b8f46 thru 0x12c381b (0xa8d6 bytes) 2024-09-13T05:02:13.452 INFO:teuthology.orchestra.run.smithi053.stdout:301 trunc from 0xa994525 to 0xb12eb61 2024-09-13T05:02:13.456 INFO:teuthology.orchestra.run.smithi053.stdout:302 punch from 0x5aff250 to 0x5b01466, (0x2216 bytes) 2024-09-13T05:02:13.457 INFO:teuthology.orchestra.run.smithi053.stdout:303 read 0x96ca537 thru 0x96cd689 (0x3153 bytes) 2024-09-13T05:02:13.457 INFO:teuthology.orchestra.run.smithi053.stdout:308 write 0x99705ef thru 0x9975235 (0x4c47 bytes) 2024-09-13T05:02:13.460 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:13.460 INFO:teuthology.orchestra.run.smithi053.stdout:311 trunc from 0xb12eb61 to 0xaf38b6b 2024-09-13T05:02:13.465 INFO:teuthology.orchestra.run.smithi053.stdout:313 clone 3 order 20 su 32768 sc 3 2024-09-13T05:02:13.485 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:13.485+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:13.509 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:13.509 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:13.509 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:13.509 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:13.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:13.601 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:13.601+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:13.664 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:13.665 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:13.673 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:13.714 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:13.714+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:13.787 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:13.787+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:13.820 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:882 3.4 e24/19 3:229424df:::rbd_header.117ca06e531b:head v 24'22, mlcod=23'20)", 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.898509+0000", 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.92166243400000003, 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072131799999999998, 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898509+0000", 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898509+0000", 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898509+0000", 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898515+0000", 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5010000000000001e-06 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898530+0000", 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4831000000000001e-05 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898541+0000", 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0703999999999999e-05 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898587+0000", 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5905000000000002e-05 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.898613+0000", 2024-09-13T05:02:13.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6157e-05 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.899198+0000", 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00058527899999999996 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.899230+0000", 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2230000000000001e-05 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:896 3.4 e24/19 3:229424df:::rbd_header.117ca06e531b:head v 24'23, mlcod=23'21)", 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.908972+0000", 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.91119996999999997, 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052920400000000002, 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908972+0000", 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908972+0000", 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908970+0000", 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908982+0000", 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1141000000000001e-05 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.908998+0000", 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6857000000000001e-05 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909008+0000", 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8560000000000002e-06 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909059+0000", 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0880000000000001e-05 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909082+0000", 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2889000000000001e-05 2024-09-13T05:02:13.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909466+0000", 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038416900000000001 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.909501+0000", 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4579000000000002e-05 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:930 3.0 e24/19 3:04479396:::rbd_id.image_client.0-clone2:head v 24'9, mlcod=22'7)", 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.931213+0000", 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.88895898799999995, 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00083496099999999995, 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931213+0000", 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931213+0000", 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931211+0000", 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931220+0000", 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9199999999999993e-06 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931235+0000", 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4970000000000001e-05 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931241+0000", 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1870000000000002e-06 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931292+0000", 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0439000000000003e-05 2024-09-13T05:02:13.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.931319+0000", 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6914e-05 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.932021+0000", 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070245799999999999 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.932048+0000", 2024-09-13T05:02:13.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6237e-05 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:965 3.4 e24/19 3:229424df:::rbd_header.117ca06e531b:head v 24'24, mlcod=24'22)", 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.081099+0000", 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.73907292400000002, 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00057929800000000001, 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081099+0000", 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081099+0000", 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081097+0000", 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:13.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081109+0000", 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1423999999999999e-05 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081125+0000", 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6021000000000001e-05 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081132+0000", 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7889999999999997e-06 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081181+0000", 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9132999999999999e-05 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081208+0000", 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7591000000000001e-05 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081654+0000", 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044576700000000002 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.081678+0000", 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3830000000000001e-05 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.839 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:967 3.4 e24/19 3:229424df:::rbd_header.117ca06e531b:head v 24'25, mlcod=24'23)", 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.082333+0000", 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.73783822399999999, 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048111600000000002, 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082333+0000", 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082333+0000", 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082333+0000", 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082338+0000", 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082347+0000", 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2399999999999996e-06 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082351+0000", 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0450000000000002e-06 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082388+0000", 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6865e-05 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082405+0000", 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7280000000000001e-05 2024-09-13T05:02:13.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082792+0000", 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038734899999999999 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.082814+0000", 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2172999999999999e-05 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1360 3.0 e24/19 3:03d65063:::rbd_data.117c4492c923.0000000000000004:head v 24'10, mlcod=22'8)", 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.387547+0000", 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.43262490999999997, 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00063643800000000002, 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387547+0000", 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387547+0000", 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387545+0000", 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387555+0000", 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1390000000000001e-06 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387571+0000", 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6725999999999999e-05 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387578+0000", 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3550000000000001e-06 2024-09-13T05:02:13.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387626+0000", 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8757000000000001e-05 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387645+0000", 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8698999999999998e-05 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.388152+0000", 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050718099999999999 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.388183+0000", 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0828999999999999e-05 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1382 3.4 e24/19 3:276c2656:::rbd_data.117c4492c923.0000000000000011:head v 24'26, mlcod=24'24)", 2024-09-13T05:02:13.849 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.408920+0000", 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.41125113499999999, 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094919399999999999, 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408920+0000", 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408920+0000", 2024-09-13T05:02:13.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408919+0000", 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408971+0000", 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2762000000000001e-05 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408992+0000", 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0547000000000002e-05 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409003+0000", 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1099e-05 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409044+0000", 2024-09-13T05:02:13.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0518000000000001e-05 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409061+0000", 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7221e-05 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409830+0000", 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076902199999999996 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409870+0000", 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9855999999999999e-05 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1393 3.4 e24/19 3:276c2656:::rbd_data.117c4492c923.0000000000000011:head v 24'27, mlcod=24'25)", 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.421194+0000", 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.39897795600000002, 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00060661899999999997, 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.854 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421194+0000", 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421194+0000", 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421192+0000", 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421203+0000", 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0485999999999999e-05 2024-09-13T05:02:13.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421222+0000", 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9171e-05 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421231+0000", 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8680000000000005e-06 2024-09-13T05:02:13.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421282+0000", 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1592999999999999e-05 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421311+0000", 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8248000000000001e-05 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421762+0000", 2024-09-13T05:02:13.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045180099999999999 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.421800+0000", 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7858999999999998e-05 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1415 3.0 e24/19 3:03d65063:::rbd_data.117c4492c923.0000000000000004:head v 24'11, mlcod=24'9)", 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.438694+0000", 2024-09-13T05:02:13.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.381477978, 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086264399999999998, 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438694+0000", 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438694+0000", 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:13.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438692+0000", 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438726+0000", 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4073999999999999e-05 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438747+0000", 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0162000000000001e-05 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438756+0000", 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9570000000000008e-06 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438805+0000", 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8933000000000001e-05 2024-09-13T05:02:13.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438826+0000", 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1790999999999999e-05 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.439532+0000", 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070596199999999997 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:13.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.439556+0000", 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3974000000000001e-05 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:13.864 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:13.866 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:13.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:02:13.907 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:13.906+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:13.965 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone2 intact 2024-09-13T05:02:13.989 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:13.988+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:14.047 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:14.047 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:14.047 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:14.047 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:14.057 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:14.089 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:14.089+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:14.118 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-13T05:02:14.191 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:14.190+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:14.231 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:14.231 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:14.231 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:14.231 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:14.231 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:14.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:14.327 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:14.327+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:14.378 INFO:teuthology.orchestra.run.smithi053.stdout:314 write 0x1eaccc4 thru 0x1eb1679 (0x49b6 bytes) 2024-09-13T05:02:14.388 INFO:teuthology.orchestra.run.smithi053.stdout:315 writesame 0x7f7a5e2 thru 0x7f87811 data_size 0x13(0xd230 bytes) 2024-09-13T05:02:14.391 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:14.391+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:14.394 INFO:teuthology.orchestra.run.smithi053.stdout:316 punch from 0x1b9d5a4 to 0x1bac800, (0xf25c bytes) 2024-09-13T05:02:14.394 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.398 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:925 3.7 3:e337519e:::rbd_object_map.117c773772bd.0000000000000004:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:12.926510+0000", 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.469732679, 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001865993, 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 925 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926510+0000", 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926510+0000", 2024-09-13T05:02:14.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926508+0000", 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926518+0000", 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2720000000000003e-06 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926535+0000", 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7822999999999999e-05 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926544+0000", 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.935e-06 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926589+0000", 2024-09-13T05:02:14.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4140000000000001e-05 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.926593+0000", 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.928237+0000", 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001643982 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.928286+0000", 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9026000000000001e-05 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:12.928376+0000", 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0012000000000005e-05 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1124 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.328652+0000", 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0675907680000001, 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016761020000000001, 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1124 2024-09-13T05:02:14.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328652+0000", 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328652+0000", 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328651+0000", 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328656+0000", 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328665+0000", 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5470000000000002e-06 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328670+0000", 2024-09-13T05:02:14.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.831e-06 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328713+0000", 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328801+0000", 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8190999999999996e-05 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.329193+0000", 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039209299999999999 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330205+0000", 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00101213 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330242+0000", 2024-09-13T05:02:14.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7344e-05 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330293+0000", 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0152999999999999e-05 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330328+0000", 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5553000000000003e-05 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1125 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.328685+0000", 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0675575959999999, 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020037599999999998, 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1125 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328685+0000", 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328685+0000", 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328685+0000", 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328688+0000", 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3579999999999999e-06 2024-09-13T05:02:14.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328692+0000", 2024-09-13T05:02:14.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.16e-06 2024-09-13T05:02:14.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.328702+0000", 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.944e-06 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.329599+0000", 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089655599999999996 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.329650+0000", 2024-09-13T05:02:14.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1363999999999998e-05 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.329882+0000", 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00023180999999999999 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330479+0000", 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00059695400000000002 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330605+0000", 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012581800000000001 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330649+0000", 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3748999999999997e-05 2024-09-13T05:02:14.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.330689+0000", 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0417999999999998e-05 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1139 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.336307+0000", 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0599357979999999, 2024-09-13T05:02:14.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002597762, 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1139 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336307+0000", 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336307+0000", 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336305+0000", 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336316+0000", 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0821e-05 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336337+0000", 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1081999999999999e-05 2024-09-13T05:02:14.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336347+0000", 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0068000000000001e-05 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336412+0000", 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5109000000000003e-05 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336516+0000", 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010453700000000001 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336895+0000", 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037822499999999999 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338844+0000", 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019489659999999999 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338849+0000", 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2569999999999998e-06 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338877+0000", 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8172000000000001e-05 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338905+0000", 2024-09-13T05:02:14.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7656000000000001e-05 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1140 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.336800+0000", 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.059443074, 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022333240000000001, 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1140 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336800+0000", 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336800+0000", 2024-09-13T05:02:14.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336799+0000", 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336805+0000", 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2949999999999999e-06 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336817+0000", 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1654000000000001e-05 2024-09-13T05:02:14.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336824+0000", 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4959999999999999e-06 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.337310+0000", 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048614099999999999 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.337405+0000", 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5061000000000003e-05 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.337621+0000", 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021545399999999999 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338914+0000", 2024-09-13T05:02:14.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012936149999999999 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338949+0000", 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4041999999999997e-05 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338994+0000", 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5797000000000002e-05 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.339033+0000", 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8708000000000003e-05 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1141 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.438 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.336836+0000", 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.059406375, 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023620659999999999, 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1141 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336836+0000", 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336836+0000", 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336836+0000", 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336839+0000", 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.146e-06 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336851+0000", 2024-09-13T05:02:14.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.222e-05 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336856+0000", 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5600000000000004e-06 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.337956+0000", 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011002169999999999 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338043+0000", 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6744999999999997e-05 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338438+0000", 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039486600000000001 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.339098+0000", 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066027299999999996 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.339150+0000", 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2515000000000001e-05 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.339181+0000", 2024-09-13T05:02:14.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0465000000000001e-05 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.339198+0000", 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7618e-05 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1212 3.18 3:1e3d0512:::rbd_data.117c4492c923.0000000000000017:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.361965+0000", 2024-09-13T05:02:14.446 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0342777270000001, 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017464189999999999, 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1212 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.361965+0000", 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.361965+0000", 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.361964+0000", 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.361969+0000", 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2089999999999996e-06 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.361980+0000", 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0295999999999999e-05 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.361986+0000", 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3439999999999997e-06 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.362021+0000", 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5293000000000002e-05 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.362072+0000", 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0967999999999997e-05 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.362363+0000", 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029060799999999998 2024-09-13T05:02:14.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.363422+0000", 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010595870000000001 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.363660+0000", 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000237099 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.363691+0000", 2024-09-13T05:02:14.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1925999999999999e-05 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.363711+0000", 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9958000000000001e-05 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1213 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.375433+0000", 2024-09-13T05:02:14.452 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0208100019999999, 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017608470000000001, 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1213 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375433+0000", 2024-09-13T05:02:14.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375433+0000", 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375430+0000", 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375444+0000", 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3414999999999999e-05 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375466+0000", 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880999999999999e-05 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375475+0000", 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9000000000000001e-06 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375541+0000", 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5199999999999999e-05 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.375662+0000", 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000121408 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.376111+0000", 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00044944499999999998 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.376845+0000", 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00073315299999999995 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.377135+0000", 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028994300000000003 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.377166+0000", 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1749999999999999e-05 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.377194+0000", 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7262000000000001e-05 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1340 3.18 3:1f1db0ed:::rbd_data.117ca06e531b.0000000000000006:5 [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.382064+0000", 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0141790100000001, 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016846400000000001, 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1340 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382064+0000", 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382064+0000", 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382063+0000", 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:14.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382066+0000", 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2709999999999999e-06 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382072+0000", 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5010000000000001e-06 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382077+0000", 2024-09-13T05:02:14.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8940000000000001e-06 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383577+0000", 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014995060000000001 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383657+0000", 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0153000000000003e-05 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383748+0000", 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1654e-05 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1353 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.382960+0000", 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.013282947, 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00283066, 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1353 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382960+0000", 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382960+0000", 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382959+0000", 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382965+0000", 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7899999999999996e-06 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382976+0000", 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0933e-05 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382983+0000", 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383028+0000", 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5429000000000002e-05 2024-09-13T05:02:14.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383128+0000", 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9623999999999993e-05 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383508+0000", 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038008999999999997 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384757+0000", 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012493319999999999 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385727+0000", 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096999 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385754+0000", 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6673e-05 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385790+0000", 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6804999999999997e-05 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1354 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.384001+0000", 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0122417109999999, 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027730200000000002, 2024-09-13T05:02:14.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1354 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384001+0000", 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384001+0000", 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384000+0000", 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384006+0000", 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4439999999999999e-06 2024-09-13T05:02:14.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384022+0000", 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6226e-05 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384030+0000", 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2390000000000002e-06 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384074+0000", 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4162000000000002e-05 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384149+0000", 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5026999999999998e-05 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384460+0000", 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031047799999999999 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386702+0000", 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022419010000000001 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386712+0000", 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0018e-05 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386748+0000", 2024-09-13T05:02:14.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6044000000000003e-05 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386774+0000", 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6285999999999999e-05 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1355 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.384809+0000", 2024-09-13T05:02:14.475 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.011433311, 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020724350000000001, 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1355 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384809+0000", 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384809+0000", 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384809+0000", 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384815+0000", 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384827+0000", 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2170999999999999e-05 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384834+0000", 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7669999999999998e-06 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384877+0000", 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3170999999999997e-05 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384946+0000", 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8917000000000004e-05 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385335+0000", 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038935199999999998 2024-09-13T05:02:14.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386818+0000", 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001483177 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386835+0000", 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6135999999999999e-05 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386862+0000", 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7161e-05 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386882+0000", 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0169999999999998e-05 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1356 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.481 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.385542+0000", 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 1.0107008200000001, 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015482479999999999, 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1356 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385542+0000", 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385542+0000", 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385541+0000", 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385546+0000", 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7820000000000001e-06 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385554+0000", 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4549999999999995e-06 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385559+0000", 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:02:14.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385827+0000", 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00026832400000000001 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385914+0000", 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6724000000000005e-05 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386300+0000", 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038579599999999999 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387020+0000", 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072010300000000004 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387024+0000", 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3270000000000002e-06 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387052+0000", 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7657e-05 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.387090+0000", 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8235999999999998e-05 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.487 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1372 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.400942+0000", 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.99530105000000002, 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015673869999999999, 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1372 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.400942+0000", 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.400942+0000", 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.400939+0000", 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:02:14.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.400954+0000", 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4842e-05 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.400977+0000", 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2858e-05 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.400997+0000", 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9630999999999999e-05 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.401047+0000", 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9999e-05 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.401123+0000", 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6209000000000002e-05 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.401312+0000", 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018969500000000001 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.401961+0000", 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064826499999999997 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.402447+0000", 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048619800000000002 2024-09-13T05:02:14.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.402480+0000", 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3158999999999999e-05 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.402509+0000", 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9048999999999999e-05 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1381 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.406877+0000", 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.98936531999999999, 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015395750000000001, 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1381 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406877+0000", 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406877+0000", 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406876+0000", 2024-09-13T05:02:14.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406883+0000", 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6359999999999997e-06 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406895+0000", 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2517000000000001e-05 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406901+0000", 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2090000000000002e-06 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.406950+0000", 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8940000000000002e-05 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.407051+0000", 2024-09-13T05:02:14.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101069 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.407465+0000", 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000413368 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408187+0000", 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072255099999999999 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408329+0000", 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014135299999999999 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408378+0000", 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8974000000000002e-05 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408417+0000", 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9288000000000001e-05 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.499 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1392 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.418972+0000", 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.97727051099999995, 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001716778, 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1392 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.418972+0000", 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.418972+0000", 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.418970+0000", 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999971 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.418984+0000", 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.411e-05 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.419006+0000", 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2612999999999998e-05 2024-09-13T05:02:14.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.419016+0000", 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6649999999999998e-06 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.419080+0000", 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3878000000000001e-05 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.419195+0000", 2024-09-13T05:02:14.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011558800000000001 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.419653+0000", 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00045789799999999999 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.420550+0000", 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089671499999999997 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.420614+0000", 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3658000000000006e-05 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.420644+0000", 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9918000000000001e-05 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.420689+0000", 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5463000000000002e-05 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.505 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1413 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.435155+0000", 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.96108822100000002, 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016117519999999999, 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1413 2024-09-13T05:02:14.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435155+0000", 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435155+0000", 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435154+0000", 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435159+0000", 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5090000000000001e-06 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435170+0000", 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.064e-05 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435177+0000", 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8449999999999997e-06 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435227+0000", 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0733999999999999e-05 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435331+0000", 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103578 2024-09-13T05:02:14.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.435734+0000", 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040268699999999999 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.436452+0000", 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071852600000000004 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.436677+0000", 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00022529200000000001 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.436726+0000", 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8683999999999997e-05 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.436766+0000", 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0184e-05 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.511 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1425 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.448153+0000", 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.94809021199999999, 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001644698, 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.512 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1425 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448153+0000", 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448153+0000", 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448152+0000", 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448157+0000", 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1410000000000004e-06 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448168+0000", 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1192999999999999e-05 2024-09-13T05:02:14.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448180+0000", 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2099999999999999e-05 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448225+0000", 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5225999999999997e-05 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448325+0000", 2024-09-13T05:02:14.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9395999999999995e-05 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.448723+0000", 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039871600000000002 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.449420+0000", 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00069618700000000002 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.449699+0000", 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027952800000000002 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.449751+0000", 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1619999999999997e-05 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.449797+0000", 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6485e-05 2024-09-13T05:02:14.517 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1455 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [read 0~40] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.901448+0000", 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.49479495299999998, 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016845950000000001, 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.518 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1455 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901448+0000", 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901448+0000", 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901446+0000", 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901454+0000", 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7670000000000003e-06 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901470+0000", 2024-09-13T05:02:14.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6480000000000001e-05 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901479+0000", 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0089999999999996e-06 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901527+0000", 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7377999999999999e-05 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.901531+0000", 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1609999999999996e-06 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.903031+0000", 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001499883 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.903074+0000", 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2698999999999999e-05 2024-09-13T05:02:14.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.903132+0000", 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8866999999999997e-05 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1549 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.unlock in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.073651+0000", 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.32259190900000001, 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015729089999999999, 2024-09-13T05:02:14.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1549 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073651+0000", 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073651+0000", 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073648+0000", 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999967 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073667+0000", 2024-09-13T05:02:14.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.929e-05 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073699+0000", 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1535000000000002e-05 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073713+0000", 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4008999999999999e-05 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073788+0000", 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5168999999999998e-05 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.073893+0000", 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010542799999999999 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.074088+0000", 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000194694 2024-09-13T05:02:14.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.074643+0000", 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00055556300000000002 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.075166+0000", 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052269399999999998 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.075198+0000", 2024-09-13T05:02:14.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1483999999999999e-05 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.075224+0000", 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6191999999999999e-05 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:14.529 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:14.530 INFO:teuthology.orchestra.run.smithi053.stdout:319 writesame 0xb24873e thru 0xb25545e data_size 0x1(0xcd21 bytes) 2024-09-13T05:02:14.530 INFO:teuthology.orchestra.run.smithi053.stdout:320 writesame 0x260c253 thru 0x2616117 data_size 0xb(0x9ec5 bytes) 2024-09-13T05:02:14.531 INFO:teuthology.orchestra.run.smithi053.stdout:321 read 0x2e74981 thru 0x2e811ac (0xc82c bytes) 2024-09-13T05:02:14.531 INFO:teuthology.orchestra.run.smithi053.stdout:322 write 0xa75b131 thru 0xa7609ec (0x58bc bytes) 2024-09-13T05:02:14.532 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:14.534 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.534 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.534 INFO:teuthology.orchestra.run.smithi053.stdout:326 write 0x69fbf72 thru 0x6a0b047 (0xf0d6 bytes) 2024-09-13T05:02:14.534 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.534 INFO:teuthology.orchestra.run.smithi053.stdout:328 punch from 0x6a13209 to 0x6a19e60, (0x6c57 bytes) 2024-09-13T05:02:14.534 INFO:teuthology.orchestra.run.smithi053.stdout:330 trunc from 0xb25545f to 0x7d035d3 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:331 write 0x67ee4ce thru 0x67f309b (0x4bce bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:333 read 0x6c4c7d0 thru 0x6c5b82b (0xf05c bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:337 punch from 0x2a8b365 to 0x2a8ea76, (0x3711 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:338 read 0x5790fe5 thru 0x579269b (0x16b7 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:339 read 0x6c15a0d thru 0x6c25730 (0xfd24 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:340 punch from 0x2302962 to 0x230e580, (0xbc1e bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:341 writesame 0x41acd07 thru 0x41b1385 data_size 0x1(0x467f bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:342 punch from 0x6fad7f4 to 0x6fb7785, (0x9f91 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:344 read 0x4395609 thru 0x439d6be (0x80b6 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:346 write 0xeded6de thru 0xedf24d2 (0x4df5 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:348 writesame 0x819e35a thru 0x81a6d22 data_size 0x1(0x89c9 bytes) 2024-09-13T05:02:14.535 INFO:teuthology.orchestra.run.smithi053.stdout:349 write 0x366ff68 thru 0x3677d61 (0x7dfa bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:350 trunc from 0xedf24d3 to 0x8cb90eb 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:351 punch from 0x7fa6041 to 0x7fa7e90, (0x1e4f bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:352 read 0x693ae56 thru 0x693cff6 (0x21a1 bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:353 writesame 0x406cbc0 thru 0x4077290 data_size 0x75(0xa6d1 bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:354 writesame 0xa05d860 thru 0xa05e4bf data_size 0x24(0xc60 bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:355 read 0x225d519 thru 0x226ab30 (0xd618 bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:356 read 0x70b189c thru 0x70beed9 (0xd63e bytes) 2024-09-13T05:02:14.536 INFO:teuthology.orchestra.run.smithi053.stdout:357 writesame 0x8cc2639 thru 0x8cd23b7 data_size 0x1(0xfd7f bytes) 2024-09-13T05:02:14.536 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:14.530+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:14.541 INFO:teuthology.orchestra.run.smithi053.stdout:358 read 0x6f0e5cd thru 0x6f1c974 (0xe3a8 bytes) 2024-09-13T05:02:14.542 INFO:teuthology.orchestra.run.smithi053.stdout:359 writesame 0x850f888 thru 0x851ccd8 data_size 0x25(0xd451 bytes) 2024-09-13T05:02:14.546 INFO:teuthology.orchestra.run.smithi053.stdout:360 read 0x8f6aaea thru 0x8f6ff80 (0x5497 bytes) 2024-09-13T05:02:14.546 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.556 INFO:teuthology.orchestra.run.smithi053.stdout:362 writesame 0xdea224b thru 0xdea8589 data_size 0x9(0x633f bytes) 2024-09-13T05:02:14.556 INFO:teuthology.orchestra.run.smithi053.stdout:363 punch from 0x29d4716 to 0x29d9db9, (0x56a3 bytes) 2024-09-13T05:02:14.556 INFO:teuthology.orchestra.run.smithi053.stdout:366 write 0x87df2a7 thru 0x87ec6f5 (0xd44f bytes) 2024-09-13T05:02:14.565 INFO:teuthology.orchestra.run.smithi053.stdout:368 read 0x1b2ccd thru 0x1bd2cf (0xa603 bytes) 2024-09-13T05:02:14.565 INFO:teuthology.orchestra.run.smithi053.stdout:369 read 0x17d2267 thru 0x17e0aef (0xe889 bytes) 2024-09-13T05:02:14.565 INFO:teuthology.orchestra.run.smithi053.stdout:371 punch from 0x84a7073 to 0x84b2c6c, (0xbbf9 bytes) 2024-09-13T05:02:14.565 INFO:teuthology.orchestra.run.smithi053.stdout:372 write 0x7968360 thru 0x7976772 (0xe413 bytes) 2024-09-13T05:02:14.568 INFO:teuthology.orchestra.run.smithi053.stdout:375 read 0x1597ae thru 0x1618c8 (0x811b bytes) 2024-09-13T05:02:14.568 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.568 INFO:teuthology.orchestra.run.smithi053.stdout:377 punch from 0xdb8b39a to 0xdb96fc3, (0xbc29 bytes) 2024-09-13T05:02:14.569 INFO:teuthology.orchestra.run.smithi053.stdout:378 writesame 0x8aa119f thru 0x8ab0392 data_size 0xa3(0xf1f4 bytes) 2024-09-13T05:02:14.575 INFO:teuthology.orchestra.run.smithi053.stdout:379 writesame 0xe11bc34 thru 0xe11e3b2 data_size 0x1(0x277f bytes) 2024-09-13T05:02:14.575 INFO:teuthology.orchestra.run.smithi053.stdout:380 punch from 0x8c266ce to 0x8c2811c, (0x1a4e bytes) 2024-09-13T05:02:14.575 INFO:teuthology.orchestra.run.smithi053.stdout:381 read 0x94e2ed9 thru 0x94ee356 (0xb47e bytes) 2024-09-13T05:02:14.576 INFO:teuthology.orchestra.run.smithi053.stdout:382 read 0x5c7a998 thru 0x5c8731c (0xc985 bytes) 2024-09-13T05:02:14.576 INFO:teuthology.orchestra.run.smithi053.stdout:383 writesame 0x5cbe3d0 thru 0x5cc64a9 data_size 0x1(0x80da bytes) 2024-09-13T05:02:14.589 INFO:teuthology.orchestra.run.smithi053.stdout:384 punch from 0xad98596 to 0xada2d03, (0xa76d bytes) 2024-09-13T05:02:14.589 INFO:teuthology.orchestra.run.smithi053.stdout:385 read 0xc47a96e thru 0xc47ebeb (0x427e bytes) 2024-09-13T05:02:14.589 INFO:teuthology.orchestra.run.smithi053.stdout:386 punch from 0x9f32dcf to 0x9f334fb, (0x72c bytes) 2024-09-13T05:02:14.589 INFO:teuthology.orchestra.run.smithi053.stdout:387 writesame 0xae8bd45 thru 0xae91ce0 data_size 0x3a(0x5f9c bytes) 2024-09-13T05:02:14.593 INFO:teuthology.orchestra.run.smithi053.stdout:389 read 0x4e46896 thru 0x4e5394d (0xd0b8 bytes) 2024-09-13T05:02:14.593 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.599 INFO:teuthology.orchestra.run.smithi053.stdout:391 trunc from 0xe11e3b3 to 0x88883d0 2024-09-13T05:02:14.610 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.619 INFO:teuthology.orchestra.run.smithi053.stdout:394 write 0xbaae3e7 thru 0xbabb346 (0xcf60 bytes) 2024-09-13T05:02:14.628 INFO:teuthology.orchestra.run.smithi053.stdout:395 write 0xbe2193d thru 0xbe2a6c9 (0x8d8d bytes) 2024-09-13T05:02:14.632 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:14.631+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:14.632 INFO:teuthology.orchestra.run.smithi053.stdout:396 read 0x49575a3 thru 0x49591be (0x1c1c bytes) 2024-09-13T05:02:14.632 INFO:teuthology.orchestra.run.smithi053.stdout:397 writesame 0x5566b07 thru 0x55698a8 data_size 0x3b(0x2da2 bytes) 2024-09-13T05:02:14.635 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:14.646 INFO:teuthology.orchestra.run.smithi053.stdout:399 trunc from 0xbe2a6ca to 0x286a3f2 2024-09-13T05:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:14.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:2107 3.1a 3:5c3c78dd:::rbd_object_map.117cdd702390:head [call lock.assert_locked in=44b,call rbd.object_map_update in=39b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.647184+0000", 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00051906000000000001, 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00055347399999999998, 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "waiting for sub ops", 2024-09-13T05:02:14.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2107 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647184+0000", 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:14.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647184+0000", 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647183+0000", 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647187+0000", 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6129999999999997e-06 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647198+0000", 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0233e-05 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647203+0000", 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0989999999999998e-06 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647262+0000", 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9027999999999998e-05 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647311+0000", 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9597e-05 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.647505+0000", 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019366899999999999 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-13T05:02:14.652 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:14.659 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:14.671 INFO:teuthology.orchestra.run.smithi053.stdout:400 trunc from 0x286a3f2 to 0x3b8d8c4 2024-09-13T05:02:14.681 INFO:teuthology.orchestra.run.smithi053.stdout:401 writesame 0x4718897 thru 0x471feb6 data_size 0x38(0x7620 bytes) 2024-09-13T05:02:14.690 INFO:teuthology.orchestra.run.smithi053.stdout:402 write 0x7615668 thru 0x7623da1 (0xe73a bytes) 2024-09-13T05:02:14.694 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:14.706 INFO:teuthology.orchestra.run.smithi053.stdout:406 write 0xdc52a1f thru 0xdc56ef7 (0x44d9 bytes) 2024-09-13T05:02:14.709 INFO:teuthology.orchestra.run.smithi053.stdout:407 write 0x4599473 thru 0x459ba0d (0x259b bytes) 2024-09-13T05:02:14.712 INFO:teuthology.orchestra.run.smithi053.stdout:409 writesame 0x3d10f3b thru 0x3d1f7a0 data_size 0xc5(0xe866 bytes) 2024-09-13T05:02:14.716 INFO:teuthology.orchestra.run.smithi053.stdout:410 write 0xc9c0093 thru 0xc9c7e9e (0x7e0c bytes) 2024-09-13T05:02:14.728 INFO:teuthology.orchestra.run.smithi053.stdout:413 trunc from 0xdc56ef8 to 0x5f2f784 2024-09-13T05:02:14.733 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:14.732+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:14.737 INFO:teuthology.orchestra.run.smithi053.stdout:415 read 0x4bff16d thru 0x4bffbef (0xa83 bytes) 2024-09-13T05:02:14.737 INFO:teuthology.orchestra.run.smithi053.stdout:416 read 0x16f1c65 thru 0x16f25dc (0x978 bytes) 2024-09-13T05:02:14.741 INFO:teuthology.orchestra.run.smithi053.stdout:417 trunc from 0x5f2f784 to 0x2634e23 2024-09-13T05:02:14.768 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:14.769 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:14.776 INFO:teuthology.orchestra.run.smithi053.stdout:418 writesame 0xc6f928a thru 0xc6fbf41 data_size 0x36(0x2cb8 bytes) 2024-09-13T05:02:14.776 INFO:teuthology.orchestra.run.smithi053.stdout:419 write 0xc5a1650 thru 0xc5a7830 (0x61e1 bytes) 2024-09-13T05:02:14.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:14.779 INFO:teuthology.orchestra.run.smithi053.stdout:420 write 0x9f78cd7 thru 0x9f85a29 (0xcd53 bytes) 2024-09-13T05:02:14.783 INFO:teuthology.orchestra.run.smithi053.stdout:421 punch from 0x6af19c9 to 0x6af8e1b, (0x7452 bytes) 2024-09-13T05:02:14.783 INFO:teuthology.orchestra.run.smithi053.stdout:422 writesame 0x6635523 thru 0x664482c data_size 0xd(0xf30a bytes) 2024-09-13T05:02:14.787 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:14.787 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:14.788 INFO:teuthology.orchestra.run.smithi053.stdout:423 read 0xb5467a1 thru 0xb547828 (0x1088 bytes) 2024-09-13T05:02:14.789 INFO:teuthology.orchestra.run.smithi053.stdout:424 clone 4 order 22 su 65536 sc 3 2024-09-13T05:02:14.804 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:14.805 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:14.820 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:14.820 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:14.834 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:14.833+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:14.836 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:14.836 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:14.853 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:14.853 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:14.868 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:02:14.874 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:14.874 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:14.882 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:14.892 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:14.893 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:14.893 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:14.914 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:14.914 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:14.935 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:14.935+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:14.943 INFO:teuthology.orchestra.run.smithi053.stdout:flattening image image_client.0-clone3 2024-09-13T05:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.005 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.013 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:15.036 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:15.036+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:15.137 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:15.136+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.156 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.165 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:15.237 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:15.237+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:15.309 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.309 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.318 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:15.361 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:15.361+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:15.364 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-13T05:02:15.421 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.421 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.430 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:15.439 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:15.439+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:15.540 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:15.540+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:15.552 INFO:teuthology.orchestra.run.smithi053.stdout:426 read 0x80c19c5 thru 0x80c4c64 (0x32a0 bytes) 2024-09-13T05:02:15.553 INFO:teuthology.orchestra.run.smithi053.stdout:427 flatten 2024-09-13T05:02:15.585 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.586 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:15.598 INFO:teuthology.orchestra.run.smithi053.stdout:429 write 0x272cdef thru 0x2731f24 (0x5136 bytes) 2024-09-13T05:02:15.600 INFO:teuthology.orchestra.run.smithi053.stdout:430 writesame 0x1271f40 thru 0x127df0b data_size 0x32(0xbfcc bytes) 2024-09-13T05:02:15.602 INFO:teuthology.orchestra.run.smithi053.stdout:431 read 0xb13163 thru 0xb19eb6 (0x6d54 bytes) 2024-09-13T05:02:15.602 INFO:teuthology.orchestra.run.smithi053.stdout:432 read 0x2ac991b thru 0x2ad2099 (0x877f bytes) 2024-09-13T05:02:15.602 INFO:teuthology.orchestra.run.smithi053.stdout:433 read 0xc62e4a8 thru 0xc63b643 (0xd19c bytes) 2024-09-13T05:02:15.603 INFO:teuthology.orchestra.run.smithi053.stdout:434 write 0x9936e24 thru 0x993710a (0x2e7 bytes) 2024-09-13T05:02:15.606 INFO:teuthology.orchestra.run.smithi053.stdout:435 write 0x538cfb6 thru 0x539cb19 (0xfb64 bytes) 2024-09-13T05:02:15.609 INFO:teuthology.orchestra.run.smithi053.stdout:436 writesame 0xdd065f thru 0xdd0f15 data_size 0x1(0x8b7 bytes) 2024-09-13T05:02:15.612 INFO:teuthology.orchestra.run.smithi053.stdout:438 read 0xa535a79 thru 0xa53d9dd (0x7f65 bytes) 2024-09-13T05:02:15.612 INFO:teuthology.orchestra.run.smithi053.stdout:439 write 0x468d045 thru 0x468fe26 (0x2de2 bytes) 2024-09-13T05:02:15.624 INFO:teuthology.orchestra.run.smithi053.stdout:440 trunc from 0xc6fbf42 to 0x4a7f15e 2024-09-13T05:02:15.641 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:15.640+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:15.644 INFO:teuthology.orchestra.run.smithi053.stdout:442 writesame 0x8a09f92 thru 0x8a19a2c data_size 0x8d(0xfa9b bytes) 2024-09-13T05:02:15.645 INFO:teuthology.orchestra.run.smithi053.stdout:444 write 0x5fc9366 thru 0x5fcff91 (0x6c2c bytes) 2024-09-13T05:02:15.656 INFO:teuthology.orchestra.run.smithi053.stdout:447 trunc from 0x8a19a2d to 0x2eb03a 2024-09-13T05:02:15.668 INFO:teuthology.orchestra.run.smithi053.stdout:448 read 0x2e83b7 thru 0x2eb039 (0x2c83 bytes) 2024-09-13T05:02:15.688 INFO:teuthology.orchestra.run.smithi053.stdout:449 write 0xcf6b3ee thru 0xcf7ab3a (0xf74d bytes) 2024-09-13T05:02:15.691 INFO:teuthology.orchestra.run.smithi053.stdout:450 write 0xa1d9031 thru 0xa1d94b3 (0x483 bytes) 2024-09-13T05:02:15.701 INFO:teuthology.orchestra.run.smithi053.stdout:454 trunc from 0xcf7ab3b to 0x59c2294 2024-09-13T05:02:15.704 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.705 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.714 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:15.718 INFO:teuthology.orchestra.run.smithi053.stdout:455 write 0x811ac0b thru 0x811d15b (0x2551 bytes) 2024-09-13T05:02:15.721 INFO:teuthology.orchestra.run.smithi053.stdout:457 write 0x39279ac thru 0x39336e7 (0xbd3c bytes) 2024-09-13T05:02:15.736 INFO:teuthology.orchestra.run.smithi053.stdout:458 write 0xccdcd91 thru 0xcce0178 (0x33e8 bytes) 2024-09-13T05:02:15.739 INFO:teuthology.orchestra.run.smithi053.stdout:459 writesame 0xba843ee thru 0xba8ba44 data_size 0x49(0x7657 bytes) 2024-09-13T05:02:15.742 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:15.741+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:15.745 INFO:teuthology.orchestra.run.smithi053.stdout:460 trunc from 0xcce0179 to 0xa20082a 2024-09-13T05:02:15.754 INFO:teuthology.orchestra.run.smithi053.stdout:461 read 0x47276a9 thru 0x4734cca (0xd622 bytes) 2024-09-13T05:02:15.759 INFO:teuthology.orchestra.run.smithi053.stdout:463 writesame 0xa39696e thru 0xa39f67a data_size 0x1(0x8d0d bytes) 2024-09-13T05:02:15.763 INFO:teuthology.orchestra.run.smithi053.stdout:465 trunc from 0xa39f67b to 0x72ee389 2024-09-13T05:02:15.772 INFO:teuthology.orchestra.run.smithi053.stdout:466 punch from 0x2efe396 to 0x2f08209, (0x9e73 bytes) 2024-09-13T05:02:15.772 INFO:teuthology.orchestra.run.smithi053.stdout:468 punch from 0x63521c7 to 0x6358550, (0x6389 bytes) 2024-09-13T05:02:15.773 INFO:teuthology.orchestra.run.smithi053.stdout:469 writesame 0x31058d thru 0x31ebe3 data_size 0x1(0xe657 bytes) 2024-09-13T05:02:15.806 INFO:teuthology.orchestra.run.smithi053.stdout:470 write 0x7a22b32 thru 0x7a25623 (0x2af2 bytes) 2024-09-13T05:02:15.809 INFO:teuthology.orchestra.run.smithi053.stdout:471 trunc from 0x7a25624 to 0x69523ee 2024-09-13T05:02:15.828 INFO:teuthology.orchestra.run.smithi053.stdout:473 writesame 0x996936a thru 0x99786a7 data_size 0x41(0xf33e bytes) 2024-09-13T05:02:15.828 INFO:teuthology.orchestra.run.smithi053.stdout:474 read 0x38158a1 thru 0x3822542 (0xcca2 bytes) 2024-09-13T05:02:15.836 INFO:teuthology.orchestra.run.smithi053.stdout:475 trunc from 0x99786a8 to 0x2171cf2 2024-09-13T05:02:15.842 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:15.842+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:15.845 INFO:teuthology.orchestra.run.smithi053.stdout:476 read 0x64ba4e thru 0x650aa0 (0x5053 bytes) 2024-09-13T05:02:15.860 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:15.860 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:15.865 INFO:teuthology.orchestra.run.smithi053.stdout:477 write 0xe153dff thru 0xe15778c (0x398e bytes) 2024-09-13T05:02:15.868 INFO:teuthology.orchestra.run.smithi053.stdout:479 clone 5 order 19 su 4096 sc 13 2024-09-13T05:02:15.869 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:15.944 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:15.943+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:16.013 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.013 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.022 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:16.044 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:16.044+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:16.090 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-13T05:02:16.128 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.128 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.138 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:16.145 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:16.145+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:16.246 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:16.245+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:16.253 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.253 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:16.311 INFO:teuthology.orchestra.run.smithi053.stdout:480 write 0x76a0c43 thru 0x76af2b5 (0xe673 bytes) 2024-09-13T05:02:16.327 INFO:teuthology.orchestra.run.smithi053.stdout:482 write 0x938e250 thru 0x9392a3e (0x47ef bytes) 2024-09-13T05:02:16.338 INFO:teuthology.orchestra.run.smithi053.stdout:483 write 0xe4aad5d thru 0xe4b5c31 (0xaed5 bytes) 2024-09-13T05:02:16.346 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:16.346+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:16.349 INFO:teuthology.orchestra.run.smithi053.stdout:484 writesame 0xdf5f3d1 thru 0xdf60779 data_size 0x1(0x13a9 bytes) 2024-09-13T05:02:16.359 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.360 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.360 INFO:teuthology.orchestra.run.smithi053.stdout:487 writesame 0xa2e8a20 thru 0xa2eedd4 data_size 0x5(0x63b5 bytes) 2024-09-13T05:02:16.368 INFO:teuthology.orchestra.run.smithi053.stdout:490 trunc from 0xe4b5c32 to 0xdf171c1 2024-09-13T05:02:16.429 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.429 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.434 INFO:teuthology.orchestra.run.smithi053.stdout:491 trunc from 0xdf171c1 to 0x3c39564 2024-09-13T05:02:16.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:16.447 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:16.446+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:16.461 INFO:teuthology.orchestra.run.smithi053.stdout:492 read 0x2e44d78 thru 0x2e46e8b (0x2114 bytes) 2024-09-13T05:02:16.464 INFO:teuthology.orchestra.run.smithi053.stdout:493 trunc from 0x3c39564 to 0x637dcc3 2024-09-13T05:02:16.476 INFO:teuthology.orchestra.run.smithi053.stdout:494 trunc from 0x637dcc3 to 0xa5c862 2024-09-13T05:02:16.548 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:16.547+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:16.576 INFO:teuthology.orchestra.run.smithi053.stdout:495 writesame 0x8492bba thru 0x8495568 data_size 0x1(0x29af bytes) 2024-09-13T05:02:16.581 INFO:teuthology.orchestra.run.smithi053.stdout:496 trunc from 0x8495569 to 0x44fb19a 2024-09-13T05:02:16.587 INFO:teuthology.orchestra.run.smithi053.stdout:497 punch from 0x1f78b5d to 0x1f8176e, (0x8c11 bytes) 2024-09-13T05:02:16.598 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.598 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.602 INFO:teuthology.orchestra.run.smithi053.stdout:498 write 0xcaa38bf thru 0xcaaf5a9 (0xbceb bytes) 2024-09-13T05:02:16.609 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:16.614 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.614 INFO:teuthology.orchestra.run.smithi053.stdout:502 write 0x2dfb4bb thru 0x2dffd67 (0x48ad bytes) 2024-09-13T05:02:16.620 INFO:teuthology.orchestra.run.smithi053.stdout:503 write 0x6d4154b thru 0x6d41d59 (0x80f bytes) 2024-09-13T05:02:16.628 INFO:teuthology.orchestra.run.smithi053.stdout:505 writesame 0xd2d404f thru 0xd2de1de data_size 0xb0(0xa190 bytes) 2024-09-13T05:02:16.628 INFO:teuthology.orchestra.run.smithi053.stdout:506 writesame 0x6710f5 thru 0x67a368 data_size 0x34(0x9274 bytes) 2024-09-13T05:02:16.649 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:16.648+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:16.720 INFO:teuthology.orchestra.run.smithi053.stdout:508 trunc from 0xd2de1df to 0x2a29244 2024-09-13T05:02:16.732 INFO:teuthology.orchestra.run.smithi053.stdout:truncating to largest ever: 0xee52989 2024-09-13T05:02:16.746 INFO:teuthology.orchestra.run.smithi053.stdout:509 trunc from 0x2a29244 to 0xee52989 2024-09-13T05:02:16.749 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:16.748+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:16.751 INFO:teuthology.orchestra.run.smithi053.stdout:510 read 0xe80408f thru 0xe812106 (0xe078 bytes) 2024-09-13T05:02:16.751 INFO:teuthology.orchestra.run.smithi053.stdout:511 read 0x4dfd3d5 thru 0x4e03c21 (0x684d bytes) 2024-09-13T05:02:16.751 INFO:teuthology.orchestra.run.smithi053.stdout:512 read 0x74b8dc7 thru 0x74c0fc7 (0x8201 bytes) 2024-09-13T05:02:16.751 INFO:teuthology.orchestra.run.smithi053.stdout:514 read 0x6936f4e thru 0x694495d (0xda10 bytes) 2024-09-13T05:02:16.751 INFO:teuthology.orchestra.run.smithi053.stdout:515 read 0x8b7a4f6 thru 0x8b80a01 (0x650c bytes) 2024-09-13T05:02:16.752 INFO:teuthology.orchestra.run.smithi053.stdout:517 writesame 0xaafe5df thru 0xab0ded9 data_size 0x1(0xf8fb bytes) 2024-09-13T05:02:16.755 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.756 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:16.769 INFO:teuthology.orchestra.run.smithi053.stdout:518 write 0x9ffc8fa thru 0xa00504b (0x8752 bytes) 2024-09-13T05:02:16.778 INFO:teuthology.orchestra.run.smithi053.stdout:519 punch from 0x77537a1 to 0x775c5c0, (0x8e1f bytes) 2024-09-13T05:02:16.778 INFO:teuthology.orchestra.run.smithi053.stdout:521 read 0xd2c84c6 thru 0xd2d4494 (0xbfcf bytes) 2024-09-13T05:02:16.778 INFO:teuthology.orchestra.run.smithi053.stdout:524 writesame 0xbb63cf6 thru 0xbb71036 data_size 0x9(0xd341 bytes) 2024-09-13T05:02:16.790 INFO:teuthology.orchestra.run.smithi053.stdout:525 punch from 0xd6b5b3e to 0xd6b5f52, (0x414 bytes) 2024-09-13T05:02:16.791 INFO:teuthology.orchestra.run.smithi053.stdout:526 read 0xc333890 thru 0xc338801 (0x4f72 bytes) 2024-09-13T05:02:16.791 INFO:teuthology.orchestra.run.smithi053.stdout:527 write 0xa469153 thru 0xa477bd8 (0xea86 bytes) 2024-09-13T05:02:16.808 INFO:teuthology.orchestra.run.smithi053.stdout:528 trunc from 0xee52989 to 0x988b421 2024-09-13T05:02:16.826 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.826 INFO:teuthology.orchestra.run.smithi053.stdout:531 writesame 0xb8843b thru 0xb8bc0b data_size 0xb(0x37d1 bytes) 2024-09-13T05:02:16.828 INFO:teuthology.orchestra.run.smithi053.stdout:532 punch from 0x76ba54c to 0x76bd8ea, (0x339e bytes) 2024-09-13T05:02:16.835 INFO:teuthology.orchestra.run.smithi053.stdout:533 trunc from 0x988b421 to 0x4944a64 2024-09-13T05:02:16.841 INFO:teuthology.orchestra.run.smithi053.stdout:534 writesame 0x134deb4 thru 0x1359fb8 data_size 0x5b(0xc105 bytes) 2024-09-13T05:02:16.850 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:16.849+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:16.853 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.866 INFO:teuthology.orchestra.run.smithi053.stdout:536 write 0x96e527a thru 0x96e6bdb (0x1962 bytes) 2024-09-13T05:02:16.869 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.869 INFO:teuthology.orchestra.run.smithi053.stdout:539 write 0x57281d0 thru 0x5734598 (0xc3c9 bytes) 2024-09-13T05:02:16.882 INFO:teuthology.orchestra.run.smithi053.stdout:541 write 0x64c223a thru 0x64cf326 (0xd0ed bytes) 2024-09-13T05:02:16.886 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:02:16.895 INFO:teuthology.orchestra.run.smithi053.stdout:542 trunc from 0x96e6bdc to 0x90aaa3b 2024-09-13T05:02:16.905 INFO:teuthology.orchestra.run.smithi053.stdout:544 read 0x309d075 thru 0x309eccd (0x1c59 bytes) 2024-09-13T05:02:16.905 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.906 INFO:teuthology.orchestra.run.smithi053.stdout:546 punch from 0x55043fd to 0x551328e, (0xee91 bytes) 2024-09-13T05:02:16.909 INFO:teuthology.orchestra.run.smithi053.stdout:547 trunc from 0x90aaa3b to 0xbc85984 2024-09-13T05:02:16.915 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:16.915 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:16.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:16.927 INFO:teuthology.orchestra.run.smithi053.stdout:548 trunc from 0xbc85984 to 0xadf6b5 2024-09-13T05:02:16.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:16.947 INFO:teuthology.orchestra.run.smithi053.stdout:550 read 0x89d4d1 thru 0x8a6791 (0x92c1 bytes) 2024-09-13T05:02:16.950 INFO:teuthology.orchestra.run.smithi053.stdout:551 read 0x573730 thru 0x57b1e2 (0x7ab3 bytes) 2024-09-13T05:02:16.950 INFO:teuthology.orchestra.run.smithi053.stdout:552 read 0x152ecf thru 0x1611c7 (0xe2f9 bytes) 2024-09-13T05:02:16.950 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.951 INFO:teuthology.orchestra.run.smithi053.stdout:557 read 0x8bccde thru 0x8c054b (0x386e bytes) 2024-09-13T05:02:16.951 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:16.951 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:16.950+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:16.969 INFO:teuthology.orchestra.run.smithi053.stdout:559 writesame 0xb816374 thru 0xb817284 data_size 0x13(0xf11 bytes) 2024-09-13T05:02:16.970 INFO:teuthology.orchestra.run.smithi053.stdout:560 write 0xe3477c thru 0xe3f6bf (0xaf44 bytes) 2024-09-13T05:02:16.990 INFO:teuthology.orchestra.run.smithi053.stdout:561 write 0xdcac132 thru 0xdcb5c10 (0x9adf bytes) 2024-09-13T05:02:17.002 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.006 INFO:teuthology.orchestra.run.smithi053.stdout:563 trunc from 0xdcb5c11 to 0xa21bca2 2024-09-13T05:02:17.018 INFO:teuthology.orchestra.run.smithi053.stdout:564 read 0x6782f2f thru 0x67832b8 (0x38a bytes) 2024-09-13T05:02:17.019 INFO:teuthology.orchestra.run.smithi053.stdout:565 read 0x199903c thru 0x19a1233 (0x81f8 bytes) 2024-09-13T05:02:17.019 INFO:teuthology.orchestra.run.smithi053.stdout:569 read 0x8197ead thru 0x81a1d6c (0x9ec0 bytes) 2024-09-13T05:02:17.019 INFO:teuthology.orchestra.run.smithi053.stdout:570 read 0x39912aa thru 0x399f6ad (0xe404 bytes) 2024-09-13T05:02:17.019 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.019 INFO:teuthology.orchestra.run.smithi053.stdout:572 read 0x831de2e thru 0x832768c (0x985f bytes) 2024-09-13T05:02:17.019 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.021 INFO:teuthology.orchestra.run.smithi053.stdout:575 trunc from 0xa21bca2 to 0x763d9a1 2024-09-13T05:02:17.028 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.028 INFO:teuthology.orchestra.run.smithi053.stdout:578 write 0x428995d thru 0x429845e (0xeb02 bytes) 2024-09-13T05:02:17.042 INFO:teuthology.orchestra.run.smithi053.stdout:579 punch from 0xb87c72 to 0xb8fadc, (0x7e6a bytes) 2024-09-13T05:02:17.048 INFO:teuthology.orchestra.run.smithi053.stdout:581 trunc from 0x763d9a1 to 0x26e0cf4 2024-09-13T05:02:17.052 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:17.052+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:17.061 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.069 INFO:teuthology.orchestra.run.smithi053.stdout:585 trunc from 0x26e0cf4 to 0xa113137 2024-09-13T05:02:17.076 INFO:teuthology.orchestra.run.smithi053.stdout:586 write 0x239f9e3 thru 0x23a967c (0x9c9a bytes) 2024-09-13T05:02:17.078 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:17.078 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:17.078 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T05:02:17.079 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:02:17.079 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:02:17.079 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T05:02:17.093 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:17.093 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:17.098 INFO:teuthology.orchestra.run.smithi053.stdout:587 write 0xdb4a596 thru 0xdb59609 (0xf074 bytes) 2024-09-13T05:02:17.099 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:17.099 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:17.104 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:17.123 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:17.124 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:17.127 INFO:teuthology.orchestra.run.smithi053.stdout:589 trunc from 0xdb5960a to 0x52ef60 2024-09-13T05:02:17.151 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:17.152 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:17.153 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:17.152+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:17.178 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:17.178 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:17.193 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:17.193 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T05:02:17.194 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:17.196 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:17.196 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:17.205 INFO:teuthology.orchestra.run.smithi053.stdout:590 write 0x9511ec8 thru 0x9517648 (0x5781 bytes) 2024-09-13T05:02:17.214 INFO:teuthology.orchestra.run.smithi053.stdout:592 read 0x8a6d083 thru 0x8a75f45 (0x8ec3 bytes) 2024-09-13T05:02:17.214 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.214 INFO:teuthology.orchestra.run.smithi053.stdout:594 punch from 0x8bc2a4a to 0x8bcb846, (0x8dfc bytes) 2024-09-13T05:02:17.214 INFO:teuthology.orchestra.run.smithi053.stdout:595 read 0x602269a thru 0x602384a (0x11b1 bytes) 2024-09-13T05:02:17.218 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:17.218 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:17.224 INFO:teuthology.orchestra.run.smithi053.stdout:596 write 0xc034b4f thru 0xc03ed62 (0xa214 bytes) 2024-09-13T05:02:17.240 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:17.240 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:17.245 INFO:teuthology.orchestra.run.smithi053.stdout:597 trunc from 0xc03ed63 to 0x305bdc8 2024-09-13T05:02:17.254 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:17.253+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:17.259 INFO:teuthology.orchestra.run.smithi053.stdout:599 trunc from 0x305bdc8 to 0x2ab1521 2024-09-13T05:02:17.263 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:17.263 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:17.266 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:17.266 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:17.267 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:17.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:17.288 INFO:teuthology.orchestra.run.smithi053.stdout:601 write 0xebc5953 thru 0xebd49b7 (0xf065 bytes) 2024-09-13T05:02:17.301 INFO:teuthology.orchestra.run.smithi053.stdout:602 read 0x5d9dd18 thru 0x5d9f2ff (0x15e8 bytes) 2024-09-13T05:02:17.301 INFO:teuthology.orchestra.run.smithi053.stdout:604 read 0x5e67f1f thru 0x5e74b90 (0xcc72 bytes) 2024-09-13T05:02:17.301 INFO:teuthology.orchestra.run.smithi053.stdout:605 punch from 0x6c1b5bd to 0x6c2a2f0, (0xed33 bytes) 2024-09-13T05:02:17.301 INFO:teuthology.orchestra.run.smithi053.stdout:606 write 0x2996846 thru 0x299fb2d (0x92e8 bytes) 2024-09-13T05:02:17.310 INFO:teuthology.orchestra.run.smithi053.stdout:607 write 0x7518107 thru 0x75267d1 (0xe6cb bytes) 2024-09-13T05:02:17.323 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.334 INFO:teuthology.orchestra.run.smithi053.stdout:609 trunc from 0xebd49b8 to 0x526b418 2024-09-13T05:02:17.349 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.349 INFO:teuthology.orchestra.run.smithi053.stdout:611 punch from 0x19ad38b to 0x19b9bf9, (0xc86e bytes) 2024-09-13T05:02:17.353 INFO:teuthology.orchestra.run.smithi053.stdout:613 trunc from 0x526b418 to 0x93dd6fd 2024-09-13T05:02:17.354 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:17.354+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:17.364 INFO:teuthology.orchestra.run.smithi053.stdout:614 trunc from 0x93dd6fd to 0xe402b1b 2024-09-13T05:02:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:615 read 0x9b555d0 thru 0x9b5a6fc (0x512d bytes) 2024-09-13T05:02:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.370 INFO:teuthology.orchestra.run.smithi053.stdout:617 write 0xc963ec5 thru 0xc970a8f (0xcbcb bytes) 2024-09-13T05:02:17.384 INFO:teuthology.orchestra.run.smithi053.stdout:618 writesame 0xe2898fb thru 0xe293639 data_size 0x53(0x9d3f bytes) 2024-09-13T05:02:17.395 INFO:teuthology.orchestra.run.smithi053.stdout:619 punch from 0x8beb2e to 0x8c15a0, (0x2a72 bytes) 2024-09-13T05:02:17.395 INFO:teuthology.orchestra.run.smithi053.stdout:620 read 0x553aa78 thru 0x554914f (0xe6d8 bytes) 2024-09-13T05:02:17.395 INFO:teuthology.orchestra.run.smithi053.stdout:622 writesame 0x1ff6e86 thru 0x2006da4 data_size 0x1(0xff1f bytes) 2024-09-13T05:02:17.422 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:17.422 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:17.422 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:17.422 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:17.422 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:17.425 INFO:teuthology.orchestra.run.smithi053.stdout:624 trunc from 0xe402b1b to 0x2897bcc 2024-09-13T05:02:17.435 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:17.441 INFO:teuthology.orchestra.run.smithi053.stdout:625 write 0xd806ed thru 0xd8ec53 (0xe567 bytes) 2024-09-13T05:02:17.453 INFO:teuthology.orchestra.run.smithi053.stdout:626 read 0x27fdea thru 0x286134 (0x634b bytes) 2024-09-13T05:02:17.455 INFO:teuthology.orchestra.run.smithi053.stdout:627 read 0x1a552c6 thru 0x1a56656 (0x1391 bytes) 2024-09-13T05:02:17.455 INFO:teuthology.orchestra.run.smithi053.stdout:628 write 0x2330f35 thru 0x2336346 (0x5412 bytes) 2024-09-13T05:02:17.455 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:17.454+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:17.462 INFO:teuthology.orchestra.run.smithi053.stdout:629 write 0x21a8ef5 thru 0x21aac35 (0x1d41 bytes) 2024-09-13T05:02:17.483 INFO:teuthology.orchestra.run.smithi053.stdout:630 write 0xcd5903b thru 0xcd5b130 (0x20f6 bytes) 2024-09-13T05:02:17.493 INFO:teuthology.orchestra.run.smithi053.stdout:631 write 0xd3c1c0e thru 0xd3ca993 (0x8d86 bytes) 2024-09-13T05:02:17.503 INFO:teuthology.orchestra.run.smithi053.stdout:633 writesame 0x5243daf thru 0x5245ed9 data_size 0x7(0x212b bytes) 2024-09-13T05:02:17.508 INFO:teuthology.orchestra.run.smithi053.stdout:634 read 0x8490423 thru 0x849f9d9 (0xf5b7 bytes) 2024-09-13T05:02:17.508 INFO:teuthology.orchestra.run.smithi053.stdout:635 read 0xd2c4735 thru 0xd2cff58 (0xb824 bytes) 2024-09-13T05:02:17.508 INFO:teuthology.orchestra.run.smithi053.stdout:636 read 0xc17385b thru 0xc17dbec (0xa392 bytes) 2024-09-13T05:02:17.510 INFO:teuthology.orchestra.run.smithi053.stdout:637 trunc from 0xd3ca994 to 0xc1c1a62 2024-09-13T05:02:17.522 INFO:teuthology.orchestra.run.smithi053.stdout:639 punch from 0x9774604 to 0x9779ab9, (0x54b5 bytes) 2024-09-13T05:02:17.523 INFO:teuthology.orchestra.run.smithi053.stdout:641 write 0x46b94e3 thru 0x46c1704 (0x8222 bytes) 2024-09-13T05:02:17.532 INFO:teuthology.orchestra.run.smithi053.stdout:642 punch from 0x3e3c13a to 0x3e466fc, (0xa5c2 bytes) 2024-09-13T05:02:17.532 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.532 INFO:teuthology.orchestra.run.smithi053.stdout:646 read 0x137dcf5 thru 0x1380db0 (0x30bc bytes) 2024-09-13T05:02:17.535 INFO:teuthology.orchestra.run.smithi053.stdout:648 trunc from 0xc1c1a62 to 0x98568fd 2024-09-13T05:02:17.541 INFO:teuthology.orchestra.run.smithi053.stdout:649 write 0x7823075 thru 0x7827a1f (0x49ab bytes) 2024-09-13T05:02:17.546 INFO:teuthology.orchestra.run.smithi053.stdout:651 write 0x64ff730 thru 0x650b69b (0xbf6c bytes) 2024-09-13T05:02:17.556 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:17.555+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:17.559 INFO:teuthology.orchestra.run.smithi053.stdout:652 read 0x1c63b5a thru 0x1c6dbdb (0xa082 bytes) 2024-09-13T05:02:17.559 INFO:teuthology.orchestra.run.smithi053.stdout:653 writesame 0x2250322 thru 0x225ad15 data_size 0x49(0xa9f4 bytes) 2024-09-13T05:02:17.564 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.564 INFO:teuthology.orchestra.run.smithi053.stdout:657 read 0x435aba7 thru 0x4365c80 (0xb0da bytes) 2024-09-13T05:02:17.569 INFO:teuthology.orchestra.run.smithi053.stdout:658 trunc from 0x98568fd to 0x59e11c6 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1382 3.4 e24/19 3:276c2656:::rbd_data.117c4492c923.0000000000000011:head v 24'26, mlcod=24'24)", 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.408920+0000", 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.1638773479999998, 2024-09-13T05:02:17.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094919399999999999, 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408920+0000", 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408920+0000", 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408919+0000", 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408971+0000", 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2762000000000001e-05 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.408992+0000", 2024-09-13T05:02:17.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0547000000000002e-05 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409003+0000", 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1099e-05 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409044+0000", 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0518000000000001e-05 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409061+0000", 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7221e-05 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409830+0000", 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076902199999999996 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.409870+0000", 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9855999999999999e-05 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1415 3.0 e24/19 3:03d65063:::rbd_data.117c4492c923.0000000000000004:head v 24'11, mlcod=24'9)", 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.438694+0000", 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.1341041909999996, 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086264399999999998, 2024-09-13T05:02:17.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438694+0000", 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438694+0000", 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438692+0000", 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438726+0000", 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4073999999999999e-05 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438747+0000", 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0162000000000001e-05 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438756+0000", 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9570000000000008e-06 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438805+0000", 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8933000000000001e-05 2024-09-13T05:02:17.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.438826+0000", 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1790999999999999e-05 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.439532+0000", 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070596199999999997 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.439556+0000", 2024-09-13T05:02:17.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3974000000000001e-05 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1878 3.0 e25/19 3:02080a24:::rbd_data.117cdd702390.0000000000000069:head v 25'20, mlcod=25'18)", 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.417655+0000", 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.155142444, 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010594319999999999, 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417655+0000", 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417655+0000", 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417654+0000", 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417682+0000", 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7515e-05 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417698+0000", 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6555e-05 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417707+0000", 2024-09-13T05:02:17.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4570000000000005e-06 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417756+0000", 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9200000000000003e-05 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.417777+0000", 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1155999999999999e-05 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.418672+0000", 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089487599999999996 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.418715+0000", 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2957999999999998e-05 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:1933 3.4 e25/19 3:24af6ebb:::rbd_data.117cdd702390.0000000000000068:head v 25'30, mlcod=25'28)", 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.441152+0000", 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.1316456669999999, 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00086104200000000001, 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441152+0000", 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441152+0000", 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441151+0000", 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441171+0000", 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9327e-05 2024-09-13T05:02:17.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441183+0000", 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1878e-05 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441188+0000", 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8829999999999997e-06 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441236+0000", 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8977000000000002e-05 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441261+0000", 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4592000000000001e-05 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.441987+0000", 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072554399999999995 2024-09-13T05:02:17.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.442013+0000", 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6529e-05 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:2118 3.4 e25/19 3:24af6ebb:::rbd_data.117cdd702390.0000000000000068:head v 25'32, mlcod=25'30)", 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.650826+0000", 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.921972169, 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001031445, 2024-09-13T05:02:17.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650826+0000", 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650826+0000", 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650824+0000", 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650834+0000", 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2259999999999994e-06 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650851+0000", 2024-09-13T05:02:17.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.694e-05 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650862+0000", 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1371e-05 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650937+0000", 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4783000000000006e-05 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.650961+0000", 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4525e-05 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651820+0000", 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085843699999999998 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651857+0000", 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7299999999999999e-05 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:2119 3.0 e25/19 3:02080a24:::rbd_data.117cdd702390.0000000000000069:head v 25'30, mlcod=25'28)", 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.651187+0000", 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.9216111840000001, 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001034227, 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.599 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651187+0000", 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651187+0000", 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651186+0000", 2024-09-13T05:02:17.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651193+0000", 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4339999999999997e-06 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651203+0000", 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0808000000000001e-05 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651210+0000", 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5899999999999996e-06 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651281+0000", 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0662999999999998e-05 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.651300+0000", 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9256999999999999e-05 2024-09-13T05:02:17.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.652188+0000", 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088847600000000002 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.652221+0000", 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2512999999999999e-05 2024-09-13T05:02:17.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:2139 3.4 e25/19 3:232f08d2:::rbd_data.117cdd702390.00000000000000be:head v 25'33, mlcod=25'31)", 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.654918+0000", 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.9178796010000001, 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085514499999999997, 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.604 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654918+0000", 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654918+0000", 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654917+0000", 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654926+0000", 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2310000000000002e-06 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654942+0000", 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5390999999999999e-05 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654949+0000", 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4390000000000003e-06 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.654995+0000", 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5890000000000003e-05 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.655016+0000", 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1056999999999998e-05 2024-09-13T05:02:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.655739+0000", 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072298299999999996 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.655773+0000", 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4186000000000001e-05 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:2300 3.0 e25/19 3:0427a411:::rbd_data.117cdd702390.000000000000003d:head v 25'36, mlcod=25'34)", 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.715054+0000", 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.857743777, 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00094129700000000001, 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715054+0000", 2024-09-13T05:02:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715054+0000", 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715053+0000", 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715073+0000", 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0264000000000001e-05 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715092+0000", 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9619999999999998e-05 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715101+0000", 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6370000000000005e-06 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715163+0000", 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2127999999999999e-05 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715181+0000", 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8026999999999999e-05 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715948+0000", 2024-09-13T05:02:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076674999999999996 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.715995+0000", 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7335e-05 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:2395 3.4 e25/19 3:24af6ebb:::rbd_data.117cdd702390.0000000000000068:head v 25'34, mlcod=25'32)", 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.788055+0000", 2024-09-13T05:02:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.784742896, 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00093917600000000001, 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788055+0000", 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788055+0000", 2024-09-13T05:02:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788054+0000", 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788087+0000", 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3194000000000001e-05 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788101+0000", 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4744e-05 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788112+0000", 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0779e-05 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788192+0000", 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9594999999999999e-05 2024-09-13T05:02:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788208+0000", 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5690000000000001e-05 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788954+0000", 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074667500000000003 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.788994+0000", 2024-09-13T05:02:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9848999999999997e-05 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8338 3.0 e27/19 3:0656b2ed:::rbd_data.117c6c62fa22.0000000000000126:head v 27'63, mlcod=27'61)", 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.210562+0000", 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.36223602100000002, 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013655410000000001, 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210562+0000", 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210562+0000", 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210560+0000", 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210580+0000", 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0202000000000001e-05 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210601+0000", 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0893000000000001e-05 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210610+0000", 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9779999999999994e-06 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210660+0000", 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0018999999999997e-05 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210683+0000", 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2731e-05 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.211891+0000", 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012074360000000001 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.211927+0000", 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6758000000000003e-05 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8377 3.0 e27/19 3:0656b2ed:::rbd_data.117c6c62fa22.0000000000000126:head v 27'64, mlcod=27'62)", 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.248618+0000", 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.32418003000000001, 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098204900000000003, 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248618+0000", 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248618+0000", 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248616+0000", 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248626+0000", 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4769999999999992e-06 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248645+0000", 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9352999999999999e-05 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248654+0000", 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0610000000000002e-06 2024-09-13T05:02:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248704+0000", 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0130000000000003e-05 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.248725+0000", 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1018000000000001e-05 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249566+0000", 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084041400000000005 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249600+0000", 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3935e-05 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8443 3.0 e27/19 3:0168967c:::rbd_data.117c6c62fa22.00000000000001dd:head v 27'66, mlcod=27'64)", 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.298029+0000", 2024-09-13T05:02:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.27476835999999999, 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084948000000000001, 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298029+0000", 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298029+0000", 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298028+0000", 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298051+0000", 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3227999999999999e-05 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298067+0000", 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6027000000000001e-05 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298077+0000", 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3610000000000007e-06 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298128+0000", 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1579000000000002e-05 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298153+0000", 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4409999999999998e-05 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298845+0000", 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00069265699999999995 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.298879+0000", 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3524e-05 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8444 3.0 e27/19 3:0409ad59:::rbd_data.117c6c62fa22.00000000000001de:head v 27'67, mlcod=27'64)", 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.299255+0000", 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.27354286900000002, 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010218390000000001, 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299255+0000", 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299255+0000", 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299254+0000", 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299269+0000", 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4494999999999999e-05 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299280+0000", 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0839000000000001e-05 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299286+0000", 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6460000000000004e-06 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299330+0000", 2024-09-13T05:02:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4138999999999999e-05 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.299351+0000", 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0531e-05 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.300232+0000", 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088136300000000002 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.300277+0000", 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4490000000000003e-05 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8618 3.4 e27/19 3:24508b78:::rbd_data.117c6c62fa22.000000000000003d:head v 27'58, mlcod=27'56)", 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.409242+0000", 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.163556223, 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010554130000000001, 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409242+0000", 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409242+0000", 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409240+0000", 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409263+0000", 2024-09-13T05:02:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3059999999999999e-05 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409284+0000", 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0548999999999999e-05 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409302+0000", 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.802e-05 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409366+0000", 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4508e-05 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.409397+0000", 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0471e-05 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.410267+0000", 2024-09-13T05:02:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087061500000000004 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.410297+0000", 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9499999999999999e-05 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8640 3.9 e27/19 3:94dad7fd:::rbd_data.117c6c62fa22.00000000000001be:head v 27'39, mlcod=27'37)", 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.430809+0000", 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.141989262, 2024-09-13T05:02:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012093640000000001, 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430809+0000", 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430809+0000", 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430807+0000", 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430818+0000", 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1065e-05 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430840+0000", 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1625e-05 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430850+0000", 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8630000000000003e-06 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430905+0000", 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5155000000000003e-05 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.430931+0000", 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.633e-05 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431986+0000", 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010548619999999999 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.432018+0000", 2024-09-13T05:02:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1906000000000003e-05 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8645 3.4 e27/19 3:27830336:::rbd_data.117c6c62fa22.00000000000001c3:head v 27'59, mlcod=27'57)", 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.431887+0000", 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.14091063300000001, 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087208299999999995, 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431887+0000", 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431887+0000", 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431887+0000", 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431893+0000", 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7399999999999998e-06 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431903+0000", 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6120000000000005e-06 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431909+0000", 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8839999999999999e-06 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431951+0000", 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2252999999999997e-05 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.431968+0000", 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7008e-05 2024-09-13T05:02:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.432729+0000", 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00076098300000000002 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.432759+0000", 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0230999999999999e-05 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8687 3.0 e27/19 3:03ac79a9:::rbd_data.117c6c62fa22.0000000000000021:head v 27'73, mlcod=27'71)", 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.450208+0000", 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.122589983, 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001037589, 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450208+0000", 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450208+0000", 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450206+0000", 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450220+0000", 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3583e-05 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450239+0000", 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9114999999999999e-05 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450248+0000", 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6389999999999998e-06 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450302+0000", 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3832999999999998e-05 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.450322+0000", 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0486999999999999e-05 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.451208+0000", 2024-09-13T05:02:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088592900000000004 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.451245+0000", 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7333000000000003e-05 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8758 3.9 e27/19 3:940a7951:::rbd_data.117c6c62fa22.00000000000001a2:head v 27'41, mlcod=27'39)", 2024-09-13T05:02:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.499080+0000", 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.073718051000000007, 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00095566100000000001, 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499080+0000", 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499080+0000", 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499078+0000", 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499094+0000", 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.518e-05 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499108+0000", 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4781e-05 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499115+0000", 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.46e-06 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499171+0000", 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5677999999999998e-05 2024-09-13T05:02:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.499197+0000", 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6290000000000001e-05 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.500001+0000", 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00080448200000000003 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.500035+0000", 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4078000000000001e-05 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8821 3.0 e27/19 3:03668f61:::rbd_data.117c6c62fa22.000000000000008d:head v 27'74, mlcod=27'72)", 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.531278+0000", 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.041519779, 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089415700000000002, 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531278+0000", 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531278+0000", 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531277+0000", 2024-09-13T05:02:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531289+0000", 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2490000000000001e-05 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531305+0000", 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5681000000000001e-05 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531313+0000", 2024-09-13T05:02:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7970000000000001e-06 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531367+0000", 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3575999999999997e-05 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.531389+0000", 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2595e-05 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.532131+0000", 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074202999999999997 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.532172+0000", 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1007999999999998e-05 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:8891 3.4 e27/19 3:215523bb:::rbd_data.117c6c62fa22.000000000000004c:head v 27'65, mlcod=27'63)", 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.561301+0000", 2024-09-13T05:02:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.011497103999999999, 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087241200000000004, 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561301+0000", 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561301+0000", 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561300+0000", 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561318+0000", 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8263000000000001e-05 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561334+0000", 2024-09-13T05:02:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5889e-05 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561341+0000", 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3019999999999997e-06 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561380+0000", 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9051000000000003e-05 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561399+0000", 2024-09-13T05:02:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8728000000000001e-05 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.562139+0000", 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00074042800000000001 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.562173+0000", 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3674999999999999e-05 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:17.673 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:17.674 INFO:teuthology.orchestra.run.smithi053.stdout:659 punch from 0x1302ecb to 0x1306b56, (0x3c8b bytes) 2024-09-13T05:02:17.674 INFO:teuthology.orchestra.run.smithi053.stdout:660 read 0x3a03a54 thru 0x3a0c2de (0x888b bytes) 2024-09-13T05:02:17.674 INFO:teuthology.orchestra.run.smithi053.stdout:661 read 0x31ed5d2 thru 0x31f21d2 (0x4c01 bytes) 2024-09-13T05:02:17.675 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:17.675 INFO:teuthology.orchestra.run.smithi053.stdout:663 trunc from 0x59e11c6 to 0xbcfcce0 2024-09-13T05:02:17.676 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:17.673+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:17.775 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:17.774+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:17.796 INFO:teuthology.orchestra.run.smithi053.stdout:664 write 0x9de987f thru 0x9dee4c3 (0x4c45 bytes) 2024-09-13T05:02:17.803 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.803 INFO:teuthology.orchestra.run.smithi053.stdout:666 write 0x1904b9a thru 0x19146bc (0xfb23 bytes) 2024-09-13T05:02:17.816 INFO:teuthology.orchestra.run.smithi053.stdout:667 read 0x69ec419 thru 0x69f1c1e (0x5806 bytes) 2024-09-13T05:02:17.816 INFO:teuthology.orchestra.run.smithi053.stdout:668 write 0xa116bef thru 0xa123dd4 (0xd1e6 bytes) 2024-09-13T05:02:17.824 INFO:teuthology.orchestra.run.smithi053.stdout:669 write 0x94c55ea thru 0x94cae41 (0x5858 bytes) 2024-09-13T05:02:17.832 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:17.832 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:17.833 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:17.839 INFO:teuthology.orchestra.run.smithi053.stdout:670 write 0xe17d2cc thru 0xe18b227 (0xdf5c bytes) 2024-09-13T05:02:17.844 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:17.853 INFO:teuthology.orchestra.run.smithi053.stdout:671 punch from 0xbcd8fb9 to 0xbcdd129, (0x4170 bytes) 2024-09-13T05:02:17.858 INFO:teuthology.orchestra.run.smithi053.stdout:673 trunc from 0xe18b228 to 0x8b9f45a 2024-09-13T05:02:17.876 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:17.876+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:17.882 INFO:teuthology.orchestra.run.smithi053.stdout:674 write 0xb9ae389 thru 0xb9b6de2 (0x8a5a bytes) 2024-09-13T05:02:17.901 INFO:teuthology.orchestra.run.smithi053.stdout:676 trunc from 0xb9b6de3 to 0x29a3b2a 2024-09-13T05:02:17.912 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:02:17.941 INFO:teuthology.orchestra.run.smithi053.stdout:678 write 0xed3fbe3 thru 0xed4274e (0x2b6c bytes) 2024-09-13T05:02:17.946 INFO:teuthology.orchestra.run.smithi053.stdout:681 writesame 0x695da93 thru 0x6964416 data_size 0x6(0x6984 bytes) 2024-09-13T05:02:17.955 INFO:teuthology.orchestra.run.smithi053.stdout:682 write 0xa797213 thru 0xa79960d (0x23fb bytes) 2024-09-13T05:02:17.960 INFO:teuthology.orchestra.run.smithi053.stdout:683 read 0x717d034 thru 0x71816b4 (0x4681 bytes) 2024-09-13T05:02:17.960 INFO:teuthology.orchestra.run.smithi053.stdout:685 writesame 0xdacdfa6 thru 0xdad1d5c data_size 0x25(0x3db7 bytes) 2024-09-13T05:02:17.965 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:17.966 INFO:teuthology.orchestra.run.smithi053.stdout:687 write 0x83957b9 thru 0x83a046f (0xacb7 bytes) 2024-09-13T05:02:17.977 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:17.976+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:17.978 INFO:teuthology.orchestra.run.smithi053.stdout:689 writesame 0xa3c2953 thru 0xa3c5e47 data_size 0x1(0x34f5 bytes) 2024-09-13T05:02:17.984 INFO:teuthology.orchestra.run.smithi053.stdout:690 writesame 0x249c98 thru 0x24fb82 data_size 0xb(0x5eeb bytes) 2024-09-13T05:02:17.989 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:17.989 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:17.989 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:17.991 INFO:teuthology.orchestra.run.smithi053.stdout:691 trunc from 0xed4274f to 0xb29938d 2024-09-13T05:02:18.001 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:18.005 INFO:teuthology.orchestra.run.smithi053.stdout:693 trunc from 0xb29938d to 0xe32680c 2024-09-13T05:02:18.011 INFO:teuthology.orchestra.run.smithi053.stdout:694 writesame 0x22c71f6 thru 0x22cd437 data_size 0x1(0x6242 bytes) 2024-09-13T05:02:18.015 INFO:teuthology.orchestra.run.smithi053.stdout:695 write 0x735f6e8 thru 0x7366658 (0x6f71 bytes) 2024-09-13T05:02:18.029 INFO:teuthology.orchestra.run.smithi053.stdout:696 trunc from 0xe32680c to 0x9f0afc6 2024-09-13T05:02:18.047 INFO:teuthology.orchestra.run.smithi053.stdout:697 write 0xb85e510 thru 0xb85f9fd (0x14ee bytes) 2024-09-13T05:02:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:698 read 0xb206c2f thru 0xb20f7aa (0x8b7c bytes) 2024-09-13T05:02:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.050 INFO:teuthology.orchestra.run.smithi053.stdout:702 writesame 0x4838309 thru 0x4840484 data_size 0x2(0x817c bytes) 2024-09-13T05:02:18.060 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.060 INFO:teuthology.orchestra.run.smithi053.stdout:704 trunc from 0xb85f9fe to 0xb8d9249 2024-09-13T05:02:18.065 INFO:teuthology.orchestra.run.smithi053.stdout:706 read 0x300060f thru 0x3005583 (0x4f75 bytes) 2024-09-13T05:02:18.065 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.065 INFO:teuthology.orchestra.run.smithi053.stdout:708 read 0x72b4f6a thru 0x72ba021 (0x50b8 bytes) 2024-09-13T05:02:18.066 INFO:teuthology.orchestra.run.smithi053.stdout:709 punch from 0xa06f195 to 0xa06fee9, (0xd54 bytes) 2024-09-13T05:02:18.066 INFO:teuthology.orchestra.run.smithi053.stdout:710 write 0xb7005fd thru 0xb704472 (0x3e76 bytes) 2024-09-13T05:02:18.071 INFO:teuthology.orchestra.run.smithi053.stdout:711 read 0x2b4d145 thru 0x2b4de28 (0xce4 bytes) 2024-09-13T05:02:18.074 INFO:teuthology.orchestra.run.smithi053.stdout:712 trunc from 0xb8d9249 to 0x95ff481 2024-09-13T05:02:18.078 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:18.077+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:18.084 INFO:teuthology.orchestra.run.smithi053.stdout:713 read 0x368d3b1 thru 0x36914ba (0x410a bytes) 2024-09-13T05:02:18.084 INFO:teuthology.orchestra.run.smithi053.stdout:714 punch from 0x3e6da2 to 0x3f210f, (0xb36d bytes) 2024-09-13T05:02:18.084 INFO:teuthology.orchestra.run.smithi053.stdout:715 read 0x42d4a11 thru 0x42e2a39 (0xe029 bytes) 2024-09-13T05:02:18.088 INFO:teuthology.orchestra.run.smithi053.stdout:716 trunc from 0x95ff481 to 0x5d2fd9f 2024-09-13T05:02:18.103 INFO:teuthology.orchestra.run.smithi053.stdout:717 read 0x4a8ef86 thru 0x4a93a5a (0x4ad5 bytes) 2024-09-13T05:02:18.118 INFO:teuthology.orchestra.run.smithi053.stdout:718 write 0xc1ca810 thru 0xc1ced5e (0x454f bytes) 2024-09-13T05:02:18.123 INFO:teuthology.orchestra.run.smithi053.stdout:719 read 0xa7ebfc5 thru 0xa7f020e (0x424a bytes) 2024-09-13T05:02:18.123 INFO:teuthology.orchestra.run.smithi053.stdout:720 read 0xb5900df thru 0xb5988e0 (0x8802 bytes) 2024-09-13T05:02:18.123 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.123 INFO:teuthology.orchestra.run.smithi053.stdout:722 writesame 0xb15b700 thru 0xb167e62 data_size 0x1(0xc763 bytes) 2024-09-13T05:02:18.138 INFO:teuthology.orchestra.run.smithi053.stdout:723 read 0x5cd3621 thru 0x5cde111 (0xaaf1 bytes) 2024-09-13T05:02:18.138 INFO:teuthology.orchestra.run.smithi053.stdout:728 trunc from 0xc1ced5f to 0xcbd69fc 2024-09-13T05:02:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:730 punch from 0xc2ae51c to 0xc2b5f54, (0x7a38 bytes) 2024-09-13T05:02:18.144 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.145 INFO:teuthology.orchestra.run.smithi053.stdout:733 trunc from 0xcbd69fc to 0xe390b49 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1139 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.336307+0000", 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8105082509999999, 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002597762, 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1139 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336307+0000", 2024-09-13T05:02:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336307+0000", 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336305+0000", 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336316+0000", 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0821e-05 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336337+0000", 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1081999999999999e-05 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336347+0000", 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0068000000000001e-05 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336412+0000", 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5109000000000003e-05 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336516+0000", 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010453700000000001 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.336895+0000", 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037822499999999999 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338844+0000", 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019489659999999999 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338849+0000", 2024-09-13T05:02:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2569999999999998e-06 2024-09-13T05:02:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338877+0000", 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8172000000000001e-05 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.338905+0000", 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7656000000000001e-05 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1353 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.382960+0000", 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7638553999999997, 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00283066, 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1353 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382960+0000", 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382960+0000", 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382959+0000", 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382965+0000", 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7899999999999996e-06 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382976+0000", 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0933e-05 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.382983+0000", 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0110000000000003e-06 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383028+0000", 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5429000000000002e-05 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383128+0000", 2024-09-13T05:02:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9623999999999993e-05 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.383508+0000", 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038008999999999997 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384757+0000", 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012493319999999999 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385727+0000", 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00096999 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385754+0000", 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6673e-05 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.385790+0000", 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6804999999999997e-05 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1354 3.7 3:e6ffde4e:::rbd_object_map.117c4492c923:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:13.384001+0000", 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.7628141639999999, 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027730200000000002, 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1354 2024-09-13T05:02:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384001+0000", 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384001+0000", 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384000+0000", 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384006+0000", 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4439999999999999e-06 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384022+0000", 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6226e-05 2024-09-13T05:02:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384030+0000", 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2390000000000002e-06 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384074+0000", 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4162000000000002e-05 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384149+0000", 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5026999999999998e-05 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.384460+0000", 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00031047799999999999 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386702+0000", 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022419010000000001 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386712+0000", 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0018e-05 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386748+0000", 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6044000000000003e-05 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:13.386774+0000", 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6285999999999999e-05 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1953 3.b 3:d3e47f4d:::rbd_data.117cdd702390.0000000000000082:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.469433+0000", 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.677382648, 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003182626, 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1953 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469433+0000", 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469433+0000", 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469432+0000", 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469437+0000", 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1909999999999999e-06 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469448+0000", 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1460999999999999e-05 2024-09-13T05:02:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469457+0000", 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2949999999999994e-06 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469507+0000", 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0810000000000003e-05 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.469600+0000", 2024-09-13T05:02:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2628999999999997e-05 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.470150+0000", 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00054989000000000004 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.472553+0000", 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024026130000000001 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.472567+0000", 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4464000000000001e-05 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.472594+0000", 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6815e-05 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.472615+0000", 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1348999999999998e-05 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.6289210700000001, 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:02:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:02:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:02:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.6287497650000002, 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:02:18.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:02:18.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:1992 3.8 e25/19 ondisk, result = 0)", 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.6217900319999998, 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016933153999999999, 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525028+0000", 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525038+0000", 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0042e-05 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525042+0000", 2024-09-13T05:02:18.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.51e-06 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541946+0000", 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016903400999999998 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541951+0000", 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7749999999999998e-06 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541958+0000", 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9489999999999999e-06 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:1993 3.8 e25/19 ondisk, result = 0)", 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.536834+0000", 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.6099811329999998, 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053475750000000002, 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536834+0000", 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536834+0000", 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536833+0000", 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536838+0000", 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8010000000000003e-06 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536850+0000", 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.137e-05 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536855+0000", 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1610000000000002e-06 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542070+0000", 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052155639999999998 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542079+0000", 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8910000000000001e-06 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542182+0000", 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010247900000000001 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8337 3.b 3:d7fc055a:::rbd_data.117c6c62fa22.0000000000000125:head [set-alloc-hint object_size 524288 write_size 524288,write 487424~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.209137+0000", 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.937678342, 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026374179999999999, 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8337 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209137+0000", 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209137+0000", 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209135+0000", 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209149+0000", 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3679000000000001e-05 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209169+0000", 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0063000000000001e-05 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209178+0000", 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9209999999999998e-06 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209235+0000", 2024-09-13T05:02:18.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7540000000000001e-05 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209352+0000", 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011651500000000001 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.209496+0000", 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014464799999999999 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.210276+0000", 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000779677 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.211683+0000", 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001406635 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.211734+0000", 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1869000000000001e-05 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.211774+0000", 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9836000000000003e-05 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8386 3.18 3:19557d2a:::rbd_data.117c6c62fa22.000000000000017d:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.249297+0000", 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.89751811999999997, 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025219999999999999, 2024-09-13T05:02:18.201 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8386 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249297+0000", 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249297+0000", 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249296+0000", 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249301+0000", 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4580000000000001e-06 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249383+0000", 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1302000000000002e-05 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249395+0000", 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2014999999999999e-05 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249445+0000", 2024-09-13T05:02:18.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0553999999999997e-05 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.249906+0000", 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00046040600000000001 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.250005+0000", 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9628000000000002e-05 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.250700+0000", 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00069464199999999998 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.251746+0000", 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001046052 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.251786+0000", 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0179999999999998e-05 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.251819+0000", 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2852999999999999e-05 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8609 3.8 3:133cf5e0:::rbd_data.117c6c62fa22.0000000000000034:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.398860+0000", 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.74795543799999997, 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029892730000000002, 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.207 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8609 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398860+0000", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398860+0000", 2024-09-13T05:02:18.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398854+0000", 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999943 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398867+0000", 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2463000000000001e-05 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398885+0000", 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8162e-05 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398891+0000", 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7229999999999998e-06 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.398937+0000", 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5535999999999999e-05 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.399066+0000", 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000129498 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.399482+0000", 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00041591099999999998 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.401482+0000", 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019991560000000002 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.401767+0000", 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028574399999999999 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.401808+0000", 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0926000000000001e-05 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.401849+0000", 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0859000000000003e-05 2024-09-13T05:02:18.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8614 3.18 3:1f08bab0:::rbd_data.117c6c62fa22.0000000000000039:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.404439+0000", 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.742376178, 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003445091, 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8614 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404439+0000", 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404439+0000", 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404437+0000", 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404445+0000", 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7540000000000006e-06 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404460+0000", 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5308999999999999e-05 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404470+0000", 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6059999999999992e-06 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404525+0000", 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4521999999999997e-05 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404650+0000", 2024-09-13T05:02:18.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125079 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405131+0000", 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048183200000000002 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407424+0000", 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022921370000000001 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407797+0000", 2024-09-13T05:02:18.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037335800000000001 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407845+0000", 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7549999999999997e-05 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407884+0000", 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9619000000000001e-05 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8617 3.b 3:d2699830:::rbd_data.117c6c62fa22.000000000000003c:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.405849+0000", 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.74096612500000003, 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059817250000000002, 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8617 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405849+0000", 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405849+0000", 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405848+0000", 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405854+0000", 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0299999999999999e-06 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405868+0000", 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3842999999999999e-05 2024-09-13T05:02:18.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405876+0000", 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5510000000000002e-06 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405935+0000", 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8984999999999999e-05 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.406092+0000", 2024-09-13T05:02:18.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000157365 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.406990+0000", 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089812699999999997 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.410855+0000", 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038648480000000002 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.411739+0000", 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088427600000000003 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.411797+0000", 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7354e-05 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.411831+0000", 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4258999999999998e-05 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.224 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8887 3.8 3:13f2a85c:::rbd_data.117c6c62fa22.0000000000000046:head [write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.560479+0000", 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.58633585899999996, 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0029376250000000001, 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8887 2024-09-13T05:02:18.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560479+0000", 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560479+0000", 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560478+0000", 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560487+0000", 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3669999999999997e-06 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560497+0000", 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8849999999999994e-06 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560503+0000", 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7710000000000001e-06 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560905+0000", 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00040144200000000001 2024-09-13T05:02:18.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.560975+0000", 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0333000000000006e-05 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.561098+0000", 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012304300000000001 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.562151+0000", 2024-09-13T05:02:18.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0010523640000000001 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.563373+0000", 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012219399999999999 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.563397+0000", 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.491e-05 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.563417+0000", 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9497999999999999e-05 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9143 3.7 3:e609a888:::rbd_data.117c6c62fa22.0000000000000051:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.236017532, 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062636530000000001, 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9143 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910796+0000", 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910804+0000", 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8979999999999996e-06 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910823+0000", 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8680000000000001e-05 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910832+0000", 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8249999999999994e-06 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910888+0000", 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6675000000000002e-05 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:18.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910894+0000", 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5999999999999997e-06 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915408+0000", 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00451441 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915490+0000", 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1624000000000003e-05 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915595+0000", 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105107 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916954+0000", 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013595149999999999 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916991+0000", 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6210000000000001e-05 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917024+0000", 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.345e-05 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917061+0000", 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7218999999999997e-05 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9146 3.2 3:478042f5:::rbd_data.117c6c62fa22.0000000000000054:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.911306+0000", 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.23550942399999999, 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006498083, 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9146 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911306+0000", 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911306+0000", 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911305+0000", 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911310+0000", 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7309999999999998e-06 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911325+0000", 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5079000000000001e-05 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911333+0000", 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3010000000000001e-06 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911379+0000", 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6767000000000001e-05 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911385+0000", 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8719999999999999e-06 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915997+0000", 2024-09-13T05:02:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046112080000000003 2024-09-13T05:02:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916092+0000", 2024-09-13T05:02:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5159000000000002e-05 2024-09-13T05:02:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916221+0000", 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012972799999999999 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917405+0000", 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001184037 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917705+0000", 2024-09-13T05:02:18.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029933500000000002 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917756+0000", 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1347000000000002e-05 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917804+0000", 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7707999999999998e-05 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9147 3.2 3:4074e5b1:::rbd_data.117c6c62fa22.0000000000000055:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.911375+0000", 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.23544021100000001, 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065962599999999996, 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9147 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.247 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911375+0000", 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911375+0000", 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911374+0000", 2024-09-13T05:02:18.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911378+0000", 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1280000000000001e-06 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911389+0000", 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0353999999999999e-05 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.249 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911397+0000", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5750000000000001e-06 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911481+0000", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3158000000000006e-05 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911486+0000", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3170000000000001e-06 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916525+0000", 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050393149999999999 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916597+0000", 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2086000000000001e-05 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916712+0000", 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114832 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917412+0000", 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00069963399999999998 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917899+0000", 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048759999999999998 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917941+0000", 2024-09-13T05:02:18.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2095999999999998e-05 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917971+0000", 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9825999999999999e-05 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9277 3.8 3:15cdb75b:::rbd_data.117c6c62fa22.0000000000000042:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.253 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.012024+0000", 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.13479091300000001, 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039206780000000004, 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9277 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012024+0000", 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012024+0000", 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012022+0000", 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.255 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012032+0000", 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0207000000000001e-05 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012052+0000", 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9278000000000001e-05 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012061+0000", 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6320000000000003e-06 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.256 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012127+0000", 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5908000000000007e-05 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012221+0000", 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4189000000000002e-05 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.257 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012692+0000", 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047016400000000002 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015840+0000", 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031479300000000002 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015851+0000", 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1810999999999999e-05 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015906+0000", 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4521000000000002e-05 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015945+0000", 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9054000000000003e-05 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9470 3.2 3:47424ef1:::rbd_data.117c6c62fa22.0000000000000164:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.130403+0000", 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.016412388, 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002576269, 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9470 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130403+0000", 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130403+0000", 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130401+0000", 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130409+0000", 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7109999999999995e-06 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130423+0000", 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4002000000000001e-05 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130433+0000", 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6029999999999994e-06 2024-09-13T05:02:18.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130492+0000", 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9562000000000003e-05 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.130581+0000", 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9048999999999994e-05 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:18.263 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.131028+0000", 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000447133 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.132855+0000", 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018268539999999999 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.132895+0000", 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9335999999999997e-05 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.132941+0000", 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6285000000000002e-05 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.132979+0000", 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8293000000000001e-05 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9475 3.8 3:1486ce3b:::rbd_data.117c6c62fa22.0000000000000169:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.135007+0000", 2024-09-13T05:02:18.265 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.01180808, 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032090479999999999, 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9475 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135007+0000", 2024-09-13T05:02:18.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135007+0000", 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135006+0000", 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:18.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135011+0000", 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7190000000000001e-06 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135024+0000", 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3247999999999999e-05 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135032+0000", 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.712e-06 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.268 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135082+0000", 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9910000000000002e-05 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135183+0000", 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101368 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.135679+0000", 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00049593399999999996 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.137868+0000", 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021891359999999999 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.138135+0000", 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000266987 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.138182+0000", 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6780999999999998e-05 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:18.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.138216+0000", 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4026999999999998e-05 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:18.271 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:18.272 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:18.271+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:18.272 INFO:teuthology.orchestra.run.smithi053.stdout:734 write 0x734d29d thru 0x7351165 (0x3ec9 bytes) 2024-09-13T05:02:18.272 INFO:teuthology.orchestra.run.smithi053.stdout:735 write 0x364ca1b thru 0x3654f14 (0x84fa bytes) 2024-09-13T05:02:18.274 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:736 write 0x2e84e01 thru 0x2e8e169 (0x9369 bytes) 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:737 trunc from 0xe390b49 to 0xd1fc9a2 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:738 trunc from 0xd1fc9a2 to 0x1974e1f 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:739 write 0x9d39326 thru 0x9d3a841 (0x151c bytes) 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:741 read 0x72b2a93 thru 0x72b6a19 (0x3f87 bytes) 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:743 punch from 0xf0acb4 to 0xf0ba83, (0xdcf bytes) 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:744 writesame 0xb38a9a thru 0xb3b29f data_size 0x2(0x2806 bytes) 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.275 INFO:teuthology.orchestra.run.smithi053.stdout:749 write 0x4775ad2 thru 0x478584c (0xfd7b bytes) 2024-09-13T05:02:18.276 INFO:teuthology.orchestra.run.smithi053.stdout:752 writesame 0x1da4301 thru 0x1db0044 data_size 0x2(0xbd44 bytes) 2024-09-13T05:02:18.282 INFO:teuthology.orchestra.run.smithi053.stdout:753 trunc from 0x9d3a842 to 0x454757f 2024-09-13T05:02:18.295 INFO:teuthology.orchestra.run.smithi053.stdout:754 write 0x229b9d1 thru 0x22a45a3 (0x8bd3 bytes) 2024-09-13T05:02:18.305 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.316 INFO:teuthology.orchestra.run.smithi053.stdout:757 write 0x9431394 thru 0x9437fa3 (0x6c10 bytes) 2024-09-13T05:02:18.325 INFO:teuthology.orchestra.run.smithi053.stdout:758 trunc from 0x9437fa4 to 0x8c32d2c 2024-09-13T05:02:18.335 INFO:teuthology.orchestra.run.smithi053.stdout:760 write 0x56d7fa6 thru 0x56e19ec (0x9a47 bytes) 2024-09-13T05:02:18.348 INFO:teuthology.orchestra.run.smithi053.stdout:763 read 0x16da50c thru 0x16dc8a5 (0x239a bytes) 2024-09-13T05:02:18.349 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.349 INFO:teuthology.orchestra.run.smithi053.stdout:765 read 0x28212b4 thru 0x2822ae2 (0x182f bytes) 2024-09-13T05:02:18.349 INFO:teuthology.orchestra.run.smithi053.stdout:767 writesame 0x4ff3783 thru 0x4ff39fa data_size 0x4(0x278 bytes) 2024-09-13T05:02:18.351 INFO:teuthology.orchestra.run.smithi053.stdout:768 writesame 0x5206b49 thru 0x521524d data_size 0x1(0xe705 bytes) 2024-09-13T05:02:18.369 INFO:teuthology.orchestra.run.smithi053.stdout:770 punch from 0x40e8d2d to 0x40ee112, (0x53e5 bytes) 2024-09-13T05:02:18.369 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.369 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.369 INFO:teuthology.orchestra.run.smithi053.stdout:773 write 0x8772f37 thru 0x8780b41 (0xdc0b bytes) 2024-09-13T05:02:18.373 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:18.372+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:18.383 INFO:teuthology.orchestra.run.smithi053.stdout:775 read 0x1f90f75 thru 0x1f9b2a6 (0xa332 bytes) 2024-09-13T05:02:18.384 INFO:teuthology.orchestra.run.smithi053.stdout:776 write 0x97f2d4 thru 0x987e52 (0x8b7f bytes) 2024-09-13T05:02:18.393 INFO:teuthology.orchestra.run.smithi053.stdout:777 punch from 0x5588ac2 to 0x558b833, (0x2d71 bytes) 2024-09-13T05:02:18.393 INFO:teuthology.orchestra.run.smithi053.stdout:778 punch from 0x963f64 to 0x965212, (0x12ae bytes) 2024-09-13T05:02:18.394 INFO:teuthology.orchestra.run.smithi053.stdout:780 punch from 0x68c528 to 0x69413b, (0x7c13 bytes) 2024-09-13T05:02:18.394 INFO:teuthology.orchestra.run.smithi053.stdout:782 read 0x1bb0a33 thru 0x1bbba6e (0xb03c bytes) 2024-09-13T05:02:18.394 INFO:teuthology.orchestra.run.smithi053.stdout:783 read 0x3def437 thru 0x3dff0f5 (0xfcbf bytes) 2024-09-13T05:02:18.395 INFO:teuthology.orchestra.run.smithi053.stdout:784 trunc from 0x8c32d2c to 0xa025da9 2024-09-13T05:02:18.400 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:18.400 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:18.400 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:18.400 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:18.405 INFO:teuthology.orchestra.run.smithi053.stdout:785 write 0xa47ea68 thru 0xa4895e3 (0xab7c bytes) 2024-09-13T05:02:18.409 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:18.420 INFO:teuthology.orchestra.run.smithi053.stdout:787 punch from 0x3dfe57e to 0x3e05912, (0x7394 bytes) 2024-09-13T05:02:18.420 INFO:teuthology.orchestra.run.smithi053.stdout:788 punch from 0x7bdaf00 to 0x7be16f0, (0x67f0 bytes) 2024-09-13T05:02:18.420 INFO:teuthology.orchestra.run.smithi053.stdout:792 read 0x28d89eb thru 0x28e65cd (0xdbe3 bytes) 2024-09-13T05:02:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:18.421 INFO:teuthology.orchestra.run.smithi053.stdout:794 clone 6 order 19 su 16384 sc 2 2024-09-13T05:02:18.473 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:18.473+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:18.512 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:18.512 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:18.512 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:18.513 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:18.513 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:18.520 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:18.574 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:18.573+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8559 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.251203539, 2024-09-13T05:02:18.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011244857, 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8559 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:18.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371562+0000", 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5789999999999998e-06 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371569+0000", 2024-09-13T05:02:18.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4209999999999998e-06 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371574+0000", 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6970000000000001e-06 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.380413+0000", 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0088391589999999992 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.380468+0000", 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5303000000000002e-05 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.380819+0000", 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035133799999999998 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382747+0000", 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019273579999999999 2024-09-13T05:02:18.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382751+0000", 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382780+0000", 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8225000000000002e-05 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382799+0000", 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9607e-05 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8560 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.251179898, 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011621819, 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8560 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371580+0000", 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371586+0000", 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.959e-06 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371596+0000", 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2609999999999999e-06 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.381495+0000", 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098995750000000007 2024-09-13T05:02:18.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.381578+0000", 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2659999999999998e-05 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.381817+0000", 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00023948 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382805+0000", 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00098802400000000011 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.383153+0000", 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034762200000000001 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.383180+0000", 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6961000000000002e-05 2024-09-13T05:02:18.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.383200+0000", 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9928e-05 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8608 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.226241403, 2024-09-13T05:02:18.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012637699000000001, 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8608 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396518+0000", 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9139999999999998e-06 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396521+0000", 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9780000000000001e-06 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396523+0000", 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2010000000000002e-06 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.406426+0000", 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0099030820000000006 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.406476+0000", 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9172000000000003e-05 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.406741+0000", 2024-09-13T05:02:18.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00026489699999999999 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.407340+0000", 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00059977299999999995 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409063+0000", 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017231810000000001 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409111+0000", 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7657000000000001e-05 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409154+0000", 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3059999999999998e-05 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9464 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.644 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.124251+0000", 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.49850737699999997, 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011271794999999999, 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9464 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.645 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124251+0000", 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124251+0000", 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124250+0000", 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124252+0000", 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6390000000000001e-06 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124255+0000", 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.027e-06 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124257+0000", 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2689999999999998e-06 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134051+0000", 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.009794054 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134107+0000", 2024-09-13T05:02:18.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5452999999999999e-05 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134303+0000", 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000196248 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134921+0000", 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061787599999999995 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135482+0000", 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056160499999999996 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135505+0000", 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3025000000000001e-05 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135522+0000", 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6824e-05 2024-09-13T05:02:18.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9699 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.37503267200000001, 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011145735, 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.652 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9699 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247728+0000", 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6450000000000001e-06 2024-09-13T05:02:18.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247733+0000", 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8440000000000003e-06 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247738+0000", 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000003e-06 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257135+0000", 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0093971109999999997 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257187+0000", 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2012000000000003e-05 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257436+0000", 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024863400000000002 2024-09-13T05:02:18.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258821+0000", 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013847659999999999 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258826+0000", 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4319999999999998e-06 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258854+0000", 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7676999999999999e-05 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258871+0000", 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7292e-05 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.658 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9700 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.37501573999999999, 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013001805, 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9700 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247745+0000", 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.943e-06 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247751+0000", 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247755+0000", 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5530000000000003e-06 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257845+0000", 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010090238 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257894+0000", 2024-09-13T05:02:18.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8194e-05 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258138+0000", 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000244305 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260659+0000", 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025215709999999998 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260690+0000", 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0670999999999998e-05 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260716+0000", 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6046000000000001e-05 2024-09-13T05:02:18.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260744+0000", 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.781e-05 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9701 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.37499848400000002, 2024-09-13T05:02:18.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013027680999999999, 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9701 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247762+0000", 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9119999999999998e-06 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247768+0000", 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1530000000000002e-06 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247773+0000", 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5870000000000003e-06 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258914+0000", 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011140957 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258992+0000", 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8226999999999995e-05 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.259350+0000", 2024-09-13T05:02:18.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000358625 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260686+0000", 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013354809999999999 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260750+0000", 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3718000000000002e-05 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260771+0000", 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1379999999999999e-05 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260787+0000", 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6081000000000001e-05 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.671 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9702 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247777+0000", 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.37498124500000002, 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013366486, 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9702 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247777+0000", 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247777+0000", 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247776+0000", 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247779+0000", 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5959999999999998e-06 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247785+0000", 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8690000000000001e-06 2024-09-13T05:02:18.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247789+0000", 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3789999999999999e-06 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.259882+0000", 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012092601 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.259958+0000", 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6709e-05 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260278+0000", 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031917799999999999 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260895+0000", 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061692099999999998 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.261098+0000", 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000203527 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.261125+0000", 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6667000000000001e-05 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.261143+0000", 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8383999999999999e-05 2024-09-13T05:02:18.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9847 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.28621658900000002, 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011087779000000001, 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.678 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9847 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336544+0000", 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9689999999999999e-06 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336550+0000", 2024-09-13T05:02:18.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7910000000000001e-06 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336554+0000", 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5669999999999996e-06 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.345827+0000", 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0092730469999999995 2024-09-13T05:02:18.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.345909+0000", 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2062000000000001e-05 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.346211+0000", 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030132100000000001 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347552+0000", 2024-09-13T05:02:18.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001341334 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347572+0000", 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0239999999999999e-05 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347598+0000", 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5647999999999999e-05 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347629+0000", 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1136999999999997e-05 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9872 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.270484487, 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012063077, 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9872 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:18.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352275+0000", 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5510000000000001e-06 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352278+0000", 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0690000000000002e-06 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352280+0000", 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4590000000000001e-06 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.358767+0000", 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064866180000000004 2024-09-13T05:02:18.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.358856+0000", 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9518000000000006e-05 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.359212+0000", 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035534599999999998 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.361913+0000", 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027010279999999998 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364249+0000", 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023360870000000001 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364296+0000", 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7497999999999998e-05 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364336+0000", 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0105000000000003e-05 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9873 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.27047555699999998, 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012174516999999999, 2024-09-13T05:02:18.690 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9873 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352284+0000", 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.575e-06 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352287+0000", 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0019999999999998e-06 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352289+0000", 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.48e-06 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360058+0000", 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077688089999999998 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360131+0000", 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3201000000000006e-05 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360323+0000", 2024-09-13T05:02:18.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019145499999999999 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.361938+0000", 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001615294 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364347+0000", 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024086429999999998 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364406+0000", 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9490000000000001e-05 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364457+0000", 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.079e-05 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9874 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.696 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.270461073, 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012309581999999999, 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9874 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352298+0000", 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8700000000000001e-06 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352305+0000", 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1940000000000003e-06 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352307+0000", 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.813e-06 2024-09-13T05:02:18.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360786+0000", 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084785250000000006 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360876+0000", 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0301000000000002e-05 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.361302+0000", 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042560599999999998 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364513+0000", 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032107390000000002 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364555+0000", 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2234000000000003e-05 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364585+0000", 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9833e-05 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364606+0000", 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1701000000000001e-05 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.702 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9875 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.27044805300000002, 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012340808, 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9875 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352311+0000", 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6309999999999999e-06 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352315+0000", 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352317+0000", 2024-09-13T05:02:18.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.616e-06 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362194+0000", 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098765569999999994 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362291+0000", 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7231999999999993e-05 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362637+0000", 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034592500000000002 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364549+0000", 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019119149999999999 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364612+0000", 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3373000000000005e-05 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364635+0000", 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2214000000000001e-05 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364651+0000", 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6137000000000002e-05 2024-09-13T05:02:18.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9876 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.27043644300000003, 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012573533, 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.709 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9876 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352323+0000", 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7719999999999999e-06 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352330+0000", 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1829999999999997e-06 2024-09-13T05:02:18.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352333+0000", 2024-09-13T05:02:18.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8380000000000002e-06 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.363216+0000", 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010882724 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.363315+0000", 2024-09-13T05:02:18.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8782000000000004e-05 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.363710+0000", 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039558399999999999 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364660+0000", 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00094961899999999996 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364839+0000", 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000179109 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364867+0000", 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8082999999999999e-05 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364895+0000", 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8070000000000001e-05 2024-09-13T05:02:18.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9898 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.25248094999999998, 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011213851, 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9898 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370278+0000", 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.652e-06 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370282+0000", 2024-09-13T05:02:18.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3809999999999999e-06 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370284+0000", 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3860000000000001e-06 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.378618+0000", 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0083340589999999996 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.378698+0000", 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9726000000000002e-05 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379083+0000", 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038519800000000001 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380487+0000", 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014035409999999999 2024-09-13T05:02:18.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381412+0000", 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00092494699999999997 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381456+0000", 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3918000000000002e-05 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381491+0000", 2024-09-13T05:02:18.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5244999999999999e-05 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9899 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370288+0000", 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.25247024800000001, 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012050751, 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9899 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370288+0000", 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370288+0000", 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370287+0000", 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370289+0000", 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7409999999999999e-06 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370294+0000", 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8949999999999997e-06 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370297+0000", 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5840000000000002e-06 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379565+0000", 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0092685560000000007 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379648+0000", 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3164999999999994e-05 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379948+0000", 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029975599999999998 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382279+0000", 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023305890000000001 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382284+0000", 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.254e-06 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382316+0000", 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2004000000000001e-05 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382338+0000", 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2449000000000001e-05 2024-09-13T05:02:18.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9900 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.25244979000000001, 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012108782, 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.728 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9900 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370310+0000", 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3350000000000001e-06 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370315+0000", 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7339999999999999e-06 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370319+0000", 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2209999999999999e-06 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380521+0000", 2024-09-13T05:02:18.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010201511 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380602+0000", 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1211000000000006e-05 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380959+0000", 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035688000000000001 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382362+0000", 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014032389999999999 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382369+0000", 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382400+0000", 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0814e-05 2024-09-13T05:02:18.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382417+0000", 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835000000000001e-05 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9901 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.25243607099999998, 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012481054, 2024-09-13T05:02:18.734 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9901 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370323+0000", 2024-09-13T05:02:18.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6649999999999999e-06 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370326+0000", 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370329+0000", 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5330000000000001e-06 2024-09-13T05:02:18.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381540+0000", 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011211087 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381609+0000", 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8922000000000002e-05 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381838+0000", 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022881999999999999 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382611+0000", 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00077353600000000002 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382744+0000", 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013245100000000001 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382772+0000", 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.798e-05 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382803+0000", 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1207000000000001e-05 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9982 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.216244503, 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011655957, 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9982 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406516+0000", 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2490000000000001e-06 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406522+0000", 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0240000000000003e-06 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406527+0000", 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5009999999999998e-06 2024-09-13T05:02:18.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.415355+0000", 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0088282719999999999 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.415430+0000", 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4474999999999994e-05 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.415774+0000", 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034416900000000002 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418070+0000", 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022960599999999999 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418105+0000", 2024-09-13T05:02:18.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5194999999999998e-05 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418134+0000", 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8878999999999999e-05 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418169+0000", 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5502e-05 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9983 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.216224572, 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011711048, 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9983 2024-09-13T05:02:18.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406537+0000", 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8550000000000004e-06 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406543+0000", 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.127e-06 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406548+0000", 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9030000000000003e-06 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.416447+0000", 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098995809999999993 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.416543+0000", 2024-09-13T05:02:18.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5599000000000004e-05 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.416861+0000", 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031824199999999999 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418196+0000", 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013346230000000001 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:18.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418200+0000", 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8340000000000001e-06 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418225+0000", 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5119000000000001e-05 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418244+0000", 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9556999999999999e-05 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.752 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:18.753 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:18.753 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:18.754 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:18.755 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:18.753+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:18.855 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:18.855+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:18.897 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:18.897 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:18.897 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:18.897 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:18.907 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:18.956 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:18.955+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:18.975 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone5 from 0xa4895e4 (overlap 0x52ef60) to 0x0 2024-09-13T05:02:19.055 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-13T05:02:19.057 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:19.056+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:19.066 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:19.066 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:19.075 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:19.158 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:19.157+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:19.180 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.292680698, 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002690039, 2024-09-13T05:02:19.181 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886006+0000", 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:19.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886013+0000", 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886021+0000", 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3070000000000003e-06 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886030+0000", 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4909999999999994e-06 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886071+0000", 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886076+0000", 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1679999999999997e-06 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887440+0000", 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001364142 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887556+0000", 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011576799999999999 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887726+0000", 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017057500000000001 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888284+0000", 2024-09-13T05:02:19.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00055794299999999998 2024-09-13T05:02:19.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888637+0000", 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035292599999999999 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888674+0000", 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7416000000000002e-05 2024-09-13T05:02:19.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888698+0000", 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3422999999999999e-05 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:1351 3.5 3:a1849f6f:::rbd_data.117ca06e531b.000000000000000a:5 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:13.382434+0000", 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.7962543020000004, 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023447030000000001, 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1351 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.190 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.382434+0000", 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.382434+0000", 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.382434+0000", 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.382436+0000", 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9599999999999999e-06 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.382442+0000", 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2949999999999999e-06 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.382447+0000", 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5399999999999997e-06 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.384593+0000", 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021460870000000001 2024-09-13T05:02:19.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.384674+0000", 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0966999999999999e-05 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:13.384779+0000", 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010522399999999999 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.194 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:1936 3.1c 3:3ac27fc6:::rbd_data.117cdd702390.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.7347198410000004, 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065407260000000002, 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1936 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443968+0000", 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443973+0000", 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1240000000000004e-06 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443984+0000", 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0796e-05 2024-09-13T05:02:19.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443990+0000", 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.939e-06 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.444026+0000", 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5697000000000001e-05 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.444119+0000", 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2684999999999998e-05 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.445186+0000", 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001067274 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450407+0000", 2024-09-13T05:02:19.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052206129999999998 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450454+0000", 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7132999999999997e-05 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450487+0000", 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2697999999999998e-05 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450509+0000", 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2767000000000001e-05 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:2050 3.16 3:6bc3cb69:::rbd_data.117cdd702390.000000000000005a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:19.202 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.6012599549999997, 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036278289999999999, 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2050 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577427+0000", 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577435+0000", 2024-09-13T05:02:19.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0120000000000005e-06 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577448+0000", 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577456+0000", 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9619999999999993e-06 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577506+0000", 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9738999999999999e-05 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577611+0000", 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105074 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.578226+0000", 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061466299999999999 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.580639+0000", 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024128550000000002 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.580966+0000", 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032758400000000001 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.581016+0000", 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0105000000000002e-05 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.581056+0000", 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9962000000000001e-05 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.208 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9140 3.16 3:6bbed544:::rbd_data.117c6c62fa22.000000000000004e:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.267958004, 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039799379999999997, 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.209 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9140 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910729+0000", 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910735+0000", 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2469999999999999e-06 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910746+0000", 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1048e-05 2024-09-13T05:02:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910751+0000", 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2229999999999998e-06 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910792+0000", 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1136000000000001e-05 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:19.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6119999999999998e-06 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913554+0000", 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00275632 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913600+0000", 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6162000000000003e-05 2024-09-13T05:02:19.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913697+0000", 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6904999999999995e-05 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914317+0000", 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062001999999999999 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914626+0000", 2024-09-13T05:02:19.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030938899999999997 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914674+0000", 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7221000000000001e-05 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914710+0000", 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6718999999999999e-05 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9239 3.1d 3:ba6bf161:::rbd_data.117c6c62fa22.0000000000000151:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.19733128, 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027959809999999999, 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9239 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981356+0000", 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981362+0000", 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7030000000000003e-06 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981373+0000", 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1063000000000001e-05 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981381+0000", 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2700000000000004e-06 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981418+0000", 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7049999999999999e-05 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981500+0000", 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1798999999999994e-05 2024-09-13T05:02:19.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.982080+0000", 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058020599999999999 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.983823+0000", 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017424020000000001 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984107+0000", 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028401700000000002 2024-09-13T05:02:19.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984134+0000", 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7489000000000001e-05 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984153+0000", 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9123999999999999e-05 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.221 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9278 3.1c 3:3c83d342:::rbd_data.117c6c62fa22.0000000000000043:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.1666320189999999, 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030035639999999998, 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9278 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:19.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012055+0000", 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012062+0000", 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6959999999999999e-06 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012076+0000", 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3994e-05 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012081+0000", 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2499999999999997e-06 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012125+0000", 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708000000000002e-05 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012208+0000", 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2834e-05 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012661+0000", 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045269800000000002 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014746+0000", 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020852079999999999 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014992+0000", 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024621199999999999 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.015029+0000", 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6698e-05 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.015060+0000", 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1265e-05 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9394 3.1c 3:384f0210:::rbd_data.117c6c62fa22.00000000000000d5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.0872233579999999, 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002679439, 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.227 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9394 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091464+0000", 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091471+0000", 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6490000000000002e-06 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091487+0000", 2024-09-13T05:02:19.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6529999999999999e-05 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091495+0000", 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2700000000000004e-06 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091552+0000", 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6740000000000002e-05 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091635+0000", 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2509000000000006e-05 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091739+0000", 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103915 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.092622+0000", 2024-09-13T05:02:19.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00088372799999999997 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094066+0000", 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001444047 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094109+0000", 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2416999999999997e-05 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094145+0000", 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5819999999999999e-05 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9468 3.c 3:37c3b698:::rbd_data.117c6c62fa22.0000000000000162:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.050179642, 2024-09-13T05:02:19.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030936710000000001, 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9468 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128507+0000", 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128515+0000", 2024-09-13T05:02:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4789999999999996e-06 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128533+0000", 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7227999999999998e-05 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128541+0000", 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128591+0000", 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9756999999999998e-05 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128715+0000", 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123979 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.129064+0000", 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034899799999999999 2024-09-13T05:02:19.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131179+0000", 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002115268 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131516+0000", 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033735900000000002 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131564+0000", 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7469999999999998e-05 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131602+0000", 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8892e-05 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9476 3.16 3:681506fa:::rbd_data.117c6c62fa22.000000000000016a:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.135630+0000", 2024-09-13T05:02:19.239 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.043058804, 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023599459999999999, 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9476 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135630+0000", 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135630+0000", 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135628+0000", 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135635+0000", 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2260000000000002e-06 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135647+0000", 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2465e-05 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135654+0000", 2024-09-13T05:02:19.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1910000000000003e-06 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135691+0000", 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6497e-05 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135768+0000", 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7645999999999995e-05 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:19.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.136091+0000", 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032245400000000001 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.137700+0000", 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016094569999999999 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.137930+0000", 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022977300000000001 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.137967+0000", 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7354999999999997e-05 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.137990+0000", 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2161e-05 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9879 3.1d 3:b8dde86d:::rbd_data.117c6c62fa22.000000000000009e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:19.245 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.82190390000000002, 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034236219999999999, 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9879 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356783+0000", 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356793+0000", 2024-09-13T05:02:19.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0233e-05 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356815+0000", 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2005999999999999e-05 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356825+0000", 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7540000000000001e-06 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356892+0000", 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7088999999999994e-05 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.357053+0000", 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016157300000000001 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.357490+0000", 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043711299999999999 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360070+0000", 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025790990000000001 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360143+0000", 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3672999999999997e-05 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360175+0000", 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1690999999999999e-05 2024-09-13T05:02:19.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360208+0000", 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3260999999999999e-05 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10086 3.1c 3:385e9163:::rbd_data.117c6c62fa22.0000000000000003:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.201197343, 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027722110000000001, 2024-09-13T05:02:19.251 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10086 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977490+0000", 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977496+0000", 2024-09-13T05:02:19.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6699999999999997e-06 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977510+0000", 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4073000000000001e-05 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977518+0000", 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5270000000000001e-06 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977573+0000", 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5096000000000002e-05 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977663+0000", 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0365000000000006e-05 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977847+0000", 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018395400000000001 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.979691+0000", 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018432190000000001 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980183+0000", 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049266399999999997 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980228+0000", 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4202000000000001e-05 2024-09-13T05:02:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980263+0000", 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5822999999999999e-05 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10124 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.989969+0000", 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.18871956200000001, 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002307113, 2024-09-13T05:02:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10124 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989969+0000", 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989969+0000", 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989968+0000", 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989975+0000", 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6720000000000004e-06 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990000+0000", 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4345000000000002e-05 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990477+0000", 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000477128 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990527+0000", 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9888000000000001e-05 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990603+0000", 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6550000000000004e-05 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990725+0000", 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121278 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992168+0000", 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001443663 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992206+0000", 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7388999999999997e-05 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992245+0000", 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9421000000000001e-05 2024-09-13T05:02:19.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992276+0000", 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0936000000000003e-05 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10126 3.1c 3:3c309cef:::rbd_data.117c6c62fa22.000000000000002e:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.1881891, 2024-09-13T05:02:19.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028308270000000002, 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10126 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990506+0000", 2024-09-13T05:02:19.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8189999999999992e-06 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990516+0000", 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0033e-05 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990523+0000", 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4670000000000001e-06 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990568+0000", 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4749000000000001e-05 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990650+0000", 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2464000000000002e-05 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990806+0000", 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015617499999999999 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991958+0000", 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00115196 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993253+0000", 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012944020000000001 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993296+0000", 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2957999999999998e-05 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993330+0000", 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4650000000000002e-05 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10130 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.18734483399999999, 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028473510000000001, 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.272 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10130 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991343+0000", 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991348+0000", 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8300000000000001e-06 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991361+0000", 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2405000000000001e-05 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991373+0000", 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1815e-05 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991419+0000", 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6575e-05 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991592+0000", 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172972 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991755+0000", 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162318 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992830+0000", 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010755809999999999 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994112+0000", 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012814409999999999 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994156+0000", 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3955000000000002e-05 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994191+0000", 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5475000000000002e-05 2024-09-13T05:02:19.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10163 3.c 3:30b8c9bb:::rbd_data.117c6c62fa22.000000000000008c:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.999294+0000", 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.17939460300000001, 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002310309, 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10163 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999294+0000", 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999294+0000", 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999293+0000", 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999299+0000", 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2550000000000003e-06 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999311+0000", 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1810999999999999e-05 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999318+0000", 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3390000000000004e-06 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999365+0000", 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6737000000000003e-05 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999439+0000", 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3949999999999995e-05 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.999589+0000", 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014980700000000001 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000391+0000", 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000801898 2024-09-13T05:02:19.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.001530+0000", 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011388419999999999 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.001570+0000", 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0231000000000002e-05 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.001604+0000", 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4374e-05 2024-09-13T05:02:19.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10168 3.1d 3:b8dde86d:::rbd_data.117c6c62fa22.000000000000009e:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.000704+0000", 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.17798434399999999, 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022577650000000001, 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10168 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000704+0000", 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000704+0000", 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000703+0000", 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000708+0000", 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.079e-06 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000723+0000", 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4718999999999999e-05 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000731+0000", 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0940000000000008e-06 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000773+0000", 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1288000000000002e-05 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.000846+0000", 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3706000000000002e-05 2024-09-13T05:02:19.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.001008+0000", 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000161356 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.001923+0000", 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00091549700000000003 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.002869+0000", 2024-09-13T05:02:19.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00094628599999999996 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.002916+0000", 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6788e-05 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.002962+0000", 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5723999999999998e-05 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.289 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10187 3.c 3:356439cd:::rbd_data.117c6c62fa22.00000000000000b2:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.17218314000000001, 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025044160000000002, 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10187 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.290 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006504+0000", 2024-09-13T05:02:19.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006510+0000", 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5500000000000002e-06 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006519+0000", 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3619999999999995e-06 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006526+0000", 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6279999999999999e-06 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006577+0000", 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1347999999999997e-05 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006662+0000", 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5271000000000004e-05 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006835+0000", 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173148 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007618+0000", 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00078252200000000001 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008945+0000", 2024-09-13T05:02:19.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001326767 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008976+0000", 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1466e-05 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009010+0000", 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3590999999999997e-05 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10188 3.5 3:a4803e9a:::rbd_data.117c6c62fa22.00000000000000b3:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.006616+0000", 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.172072749, 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024080159999999998, 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.296 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10188 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006616+0000", 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006616+0000", 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006615+0000", 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006621+0000", 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9220000000000002e-06 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006630+0000", 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9269999999999994e-06 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006637+0000", 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4610000000000002e-06 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006686+0000", 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9239000000000001e-05 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006775+0000", 2024-09-13T05:02:19.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9122999999999994e-05 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006954+0000", 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017872499999999999 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008254+0000", 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001299421 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008914+0000", 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066057599999999996 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008963+0000", 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8661e-05 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009024+0000", 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0921999999999997e-05 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.302 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10198 3.16 3:68a264b6:::rbd_data.117c6c62fa22.000000000000010c:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.009118+0000", 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.16957039900000001, 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023677540000000001, 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.303 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10198 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009118+0000", 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009118+0000", 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009117+0000", 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009123+0000", 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6829999999999996e-06 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009137+0000", 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4260000000000001e-05 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009149+0000", 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1524e-05 2024-09-13T05:02:19.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009188+0000", 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9437000000000002e-05 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009261+0000", 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999998e-05 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:19.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009369+0000", 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107549 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010175+0000", 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080588000000000003 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:19.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.011409+0000", 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012343499999999999 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.011451+0000", 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1956999999999998e-05 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.011486+0000", 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4829000000000001e-05 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.310 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:19.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:19.312 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:19.313 INFO:teuthology.orchestra.run.smithi053.stdout:795 punch from 0x1d3e3b6 to 0x1d4195b, (0x35a5 bytes) 2024-09-13T05:02:19.313 INFO:teuthology.orchestra.run.smithi053.stdout:798 punch from 0x3ac66be to 0x3ad3c02, (0xd544 bytes) 2024-09-13T05:02:19.313 INFO:teuthology.orchestra.run.smithi053.stdout:799 writesame 0x8886f9c thru 0x8888aaf data_size 0x2(0x1b14 bytes) 2024-09-13T05:02:19.313 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.313 INFO:teuthology.orchestra.run.smithi053.stdout:801 punch from 0x5bfc20c to 0x5c0425c, (0x8050 bytes) 2024-09-13T05:02:19.313 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.314 INFO:teuthology.orchestra.run.smithi053.stdout:803 write 0x35d7cbe thru 0x35db2b9 (0x35fc bytes) 2024-09-13T05:02:19.314 INFO:teuthology.orchestra.run.smithi053.stdout:804 punch from 0x6336573 to 0x634610b, (0xfb98 bytes) 2024-09-13T05:02:19.314 INFO:teuthology.orchestra.run.smithi053.stdout:805 punch from 0x34c9436 to 0x34d0f02, (0x7acc bytes) 2024-09-13T05:02:19.314 INFO:teuthology.orchestra.run.smithi053.stdout:806 read 0x1a550c6 thru 0x1a57e6c (0x2da7 bytes) 2024-09-13T05:02:19.314 INFO:teuthology.orchestra.run.smithi053.stdout:807 writesame 0x6b2100e thru 0x6b28c58 data_size 0x1(0x7c4b bytes) 2024-09-13T05:02:19.314 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:19.310+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:19.330 INFO:teuthology.orchestra.run.smithi053.stdout:808 writesame 0xc4277c8 thru 0xc42a02a data_size 0x1(0x2863 bytes) 2024-09-13T05:02:19.330 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.330 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.335 INFO:teuthology.orchestra.run.smithi053.stdout:812 trunc from 0xc42a02b to 0x77768f7 2024-09-13T05:02:19.350 INFO:teuthology.orchestra.run.smithi053.stdout:813 trunc from 0x77768f7 to 0x630a4bc 2024-09-13T05:02:19.366 INFO:teuthology.orchestra.run.smithi053.stdout:814 trunc from 0x630a4bc to 0x3b0588c 2024-09-13T05:02:19.378 INFO:teuthology.orchestra.run.smithi053.stdout:815 read 0x1560186 thru 0x15608cc (0x747 bytes) 2024-09-13T05:02:19.391 INFO:teuthology.orchestra.run.smithi053.stdout:816 writesame 0x980e324 thru 0x981a7ae data_size 0x1d(0xc48b bytes) 2024-09-13T05:02:19.397 INFO:teuthology.orchestra.run.smithi053.stdout:817 trunc from 0x981a7af to 0x45ccc47 2024-09-13T05:02:19.402 INFO:teuthology.orchestra.run.smithi053.stdout:818 read 0x1d786ad thru 0x1d8031f (0x7c73 bytes) 2024-09-13T05:02:19.408 INFO:teuthology.orchestra.run.smithi053.stdout:819 trunc from 0x45ccc47 to 0x8d2b104 2024-09-13T05:02:19.411 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:19.411+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:19.413 INFO:teuthology.orchestra.run.smithi053.stdout:821 punch from 0x1f9747a to 0x1fa1c06, (0xa78c bytes) 2024-09-13T05:02:19.418 INFO:teuthology.orchestra.run.smithi053.stdout:822 write 0x9028053 thru 0x902c207 (0x41b5 bytes) 2024-09-13T05:02:19.423 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.423 INFO:teuthology.orchestra.run.smithi053.stdout:825 write 0x52f8178 thru 0x5307098 (0xef21 bytes) 2024-09-13T05:02:19.429 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.429 INFO:teuthology.orchestra.run.smithi053.stdout:827 writesame 0x7f19a57 thru 0x7f20450 data_size 0x5(0x69fa bytes) 2024-09-13T05:02:19.438 INFO:teuthology.orchestra.run.smithi053.stdout:829 trunc from 0x902c208 to 0xc2ed700 2024-09-13T05:02:19.444 INFO:teuthology.orchestra.run.smithi053.stdout:830 punch from 0x4a79ce to 0x4b0818, (0x8e4a bytes) 2024-09-13T05:02:19.444 INFO:teuthology.orchestra.run.smithi053.stdout:831 write 0xae9bf33 thru 0xae9c416 (0x4e4 bytes) 2024-09-13T05:02:19.455 INFO:teuthology.orchestra.run.smithi053.stdout:833 trunc from 0xc2ed700 to 0x53790f4 2024-09-13T05:02:19.461 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:19.461 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:19.463 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:19.463 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:19.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.463 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4476.0:10611 3.1e e29/19 3:7d0f2674:::rbd_object_map.117cbf4a1622:head v 29'98, mlcod=29'96)", 2024-09-13T05:02:19.463 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.463338+0000", 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00021274799999999999, 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022795700000000001, 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463338+0000", 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:19.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463338+0000", 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463338+0000", 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463343+0000", 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.169e-06 2024-09-13T05:02:19.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463352+0000", 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5149999999999995e-06 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463358+0000", 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3619999999999997e-06 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:19.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463414+0000", 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6715999999999997e-05 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.463428+0000", 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3356999999999999e-05 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-13T05:02:19.467 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:19.468 INFO:teuthology.orchestra.run.smithi053.stdout:834 write 0x3f7b237 thru 0x3f7f78d (0x4557 bytes) 2024-09-13T05:02:19.472 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.473 INFO:teuthology.orchestra.run.smithi053.stdout:837 trunc from 0x53790f4 to 0x6dae86e 2024-09-13T05:02:19.474 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:19.481 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:19.481 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:19.491 INFO:teuthology.orchestra.run.smithi053.stdout:838 writesame 0xdf2febb thru 0xdf37ba6 data_size 0x7b(0x7cec bytes) 2024-09-13T05:02:19.491 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.491 INFO:teuthology.orchestra.run.smithi053.stdout:840 write 0xdc23bf2 thru 0xdc333ac (0xf7bb bytes) 2024-09-13T05:02:19.494 INFO:teuthology.orchestra.run.smithi053.stdout:843 read 0x75b80fb thru 0x75c6234 (0xe13a bytes) 2024-09-13T05:02:19.494 INFO:teuthology.orchestra.run.smithi053.stdout:845 read 0x52a4a09 thru 0x52aa151 (0x5749 bytes) 2024-09-13T05:02:19.499 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:19.500 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:19.506 INFO:teuthology.orchestra.run.smithi053.stdout:847 trunc from 0xdf37ba7 to 0x43adbf8 2024-09-13T05:02:19.516 INFO:teuthology.orchestra.run.smithi053.stdout:848 punch from 0x1c77695 to 0x1c8114b, (0x9ab6 bytes) 2024-09-13T05:02:19.516 INFO:teuthology.orchestra.run.smithi053.stdout:849 read 0x19c72d5 thru 0x19cb1c1 (0x3eed bytes) 2024-09-13T05:02:19.517 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:19.517+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:19.518 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:19.519 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:19.520 INFO:teuthology.orchestra.run.smithi053.stdout:850 trunc from 0x43adbf8 to 0x7722ff1 2024-09-13T05:02:19.537 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:19.537 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:19.538 INFO:teuthology.orchestra.run.smithi053.stdout:851 writesame 0xd17068c thru 0xd17482f data_size 0x2(0x41a4 bytes) 2024-09-13T05:02:19.538 INFO:teuthology.orchestra.run.smithi053.stdout:855 write 0x17a21b2 thru 0x17a7c5e (0x5aad bytes) 2024-09-13T05:02:19.557 INFO:teuthology.orchestra.run.smithi053.stdout:858 trunc from 0xd174830 to 0x4ea7567 2024-09-13T05:02:19.559 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:19.560 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:19.577 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:19.577 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:19.578 INFO:teuthology.orchestra.run.smithi053.stdout:859 writesame 0xe08944b thru 0xe096be7 data_size 0x3(0xd79d bytes) 2024-09-13T05:02:19.578 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.578 INFO:teuthology.orchestra.run.smithi053.stdout:861 write 0x7f33d2b thru 0x7f3f95e (0xbc34 bytes) 2024-09-13T05:02:19.582 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.582 INFO:teuthology.orchestra.run.smithi053.stdout:863 read 0xd2a96c0 thru 0xd2b2a10 (0x9351 bytes) 2024-09-13T05:02:19.582 INFO:teuthology.orchestra.run.smithi053.stdout:864 writesame 0x12fe6ec thru 0x1308eb5 data_size 0x3(0xa7ca bytes) 2024-09-13T05:02:19.585 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:19.586 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:19.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:19.599 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.599 INFO:teuthology.orchestra.run.smithi053.stdout:869 punch from 0x7122f6d to 0x71323fd, (0xf490 bytes) 2024-09-13T05:02:19.599 INFO:teuthology.orchestra.run.smithi053.stdout:870 write 0xb73e0b0 thru 0xb743d3b (0x5c8c bytes) 2024-09-13T05:02:19.599 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:19.600 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:19.603 INFO:teuthology.orchestra.run.smithi053.stdout:871 write 0x105926d thru 0x105d3c7 (0x415b bytes) 2024-09-13T05:02:19.615 INFO:teuthology.orchestra.run.smithi053.stdout:872 flatten 2024-09-13T05:02:19.637 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:19.636+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:19.714 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:19.713+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:19.788 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:19.789 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:19.798 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:19.842 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:19.842+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:19.859 INFO:teuthology.orchestra.run.smithi053.stdout:874 write 0x9b92c9d thru 0x9b940a0 (0x1404 bytes) 2024-09-13T05:02:19.862 INFO:teuthology.orchestra.run.smithi053.stdout:875 write 0xae08228 thru 0xae09046 (0xe1f bytes) 2024-09-13T05:02:19.866 INFO:teuthology.orchestra.run.smithi053.stdout:876 read 0x757626 thru 0x763474 (0xbe4f bytes) 2024-09-13T05:02:19.866 INFO:teuthology.orchestra.run.smithi053.stdout:877 writesame 0x2b4c4a0 thru 0x2b5afa0 data_size 0x1(0xeb01 bytes) 2024-09-13T05:02:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:878 writesame 0xe1f42a9 thru 0xe2027e8 data_size 0x83(0xe540 bytes) 2024-09-13T05:02:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:879 read 0x14c2f20 thru 0x14ccd01 (0x9de2 bytes) 2024-09-13T05:02:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:882 punch from 0xeb7db8 to 0xec1931, (0x9b79 bytes) 2024-09-13T05:02:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:884 read 0x40eca47 thru 0x40ef192 (0x274c bytes) 2024-09-13T05:02:19.889 INFO:teuthology.orchestra.run.smithi053.stdout:885 writesame 0x8370060 thru 0x837aac2 data_size 0x1(0xaa63 bytes) 2024-09-13T05:02:19.901 INFO:teuthology.orchestra.run.smithi053.stdout:886 write 0x59b961b thru 0x59c301f (0x9a05 bytes) 2024-09-13T05:02:19.905 INFO:teuthology.orchestra.run.smithi053.stdout:887 read 0x353fdc6 thru 0x3543c15 (0x3e50 bytes) 2024-09-13T05:02:19.905 INFO:teuthology.orchestra.run.smithi053.stdout:889 read 0x3d3fac1 thru 0x3d43dca (0x430a bytes) 2024-09-13T05:02:19.905 INFO:teuthology.orchestra.run.smithi053.stdout:892 writesame 0x1d30fb6 thru 0x1d39106 data_size 0x5(0x8151 bytes) 2024-09-13T05:02:19.908 INFO:teuthology.orchestra.run.smithi053.stdout:895 punch from 0x6a4e73 to 0x6b4072, (0xf1ff bytes) 2024-09-13T05:02:19.909 INFO:teuthology.orchestra.run.smithi053.stdout:896 punch from 0xd4cd924 to 0xd4dc0ba, (0xe796 bytes) 2024-09-13T05:02:19.909 INFO:teuthology.orchestra.run.smithi053.stdout:898 punch from 0x4537646 to 0x454763e, (0xfff8 bytes) 2024-09-13T05:02:19.909 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:19.909 INFO:teuthology.orchestra.run.smithi053.stdout:902 write 0xb654d05 thru 0xb66182a (0xcb26 bytes) 2024-09-13T05:02:19.915 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:19.915 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:19.916 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:19.915+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:19.924 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:19.925 INFO:teuthology.orchestra.run.smithi053.stdout:903 trunc from 0xe2027e9 to 0x2a05325 2024-09-13T05:02:19.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:02:19.938 INFO:teuthology.orchestra.run.smithi053.stdout:904 punch from 0x5d657 to 0x6c44c, (0xedf5 bytes) 2024-09-13T05:02:19.940 INFO:teuthology.orchestra.run.smithi053.stdout:905 trunc from 0x2a05325 to 0x428022b 2024-09-13T05:02:19.956 INFO:teuthology.orchestra.run.smithi053.stdout:906 trunc from 0x428022b to 0xe21436a 2024-09-13T05:02:19.963 INFO:teuthology.orchestra.run.smithi053.stdout:907 trunc from 0xe21436a to 0xba76fbc 2024-09-13T05:02:19.977 INFO:teuthology.orchestra.run.smithi053.stdout:908 write 0xe9fc6b7 thru 0xea05551 (0x8e9b bytes) 2024-09-13T05:02:19.980 INFO:teuthology.orchestra.run.smithi053.stdout:909 write 0x5507a03 thru 0x550c3cf (0x49cd bytes) 2024-09-13T05:02:19.984 INFO:teuthology.orchestra.run.smithi053.stdout:910 write 0x2cfdcc5 thru 0x2cfe30a (0x646 bytes) 2024-09-13T05:02:19.986 INFO:teuthology.orchestra.run.smithi053.stdout:911 read 0x5d4c075 thru 0x5d55bdd (0x9b69 bytes) 2024-09-13T05:02:20.003 INFO:teuthology.orchestra.run.smithi053.stdout:912 trunc from 0xea05552 to 0x269731 2024-09-13T05:02:20.017 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:20.016+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:20.028 INFO:teuthology.orchestra.run.smithi053.stdout:915 write 0x4a3d7d1 thru 0x4a3ed58 (0x1588 bytes) 2024-09-13T05:02:20.032 INFO:teuthology.orchestra.run.smithi053.stdout:916 trunc from 0x4a3ed59 to 0x6269522 2024-09-13T05:02:20.038 INFO:teuthology.orchestra.run.smithi053.stdout:917 punch from 0x1b506b to 0x1bf43f, (0xa3d4 bytes) 2024-09-13T05:02:20.038 INFO:teuthology.orchestra.run.smithi053.stdout:918 punch from 0x4fbb32f to 0x4fc142f, (0x6100 bytes) 2024-09-13T05:02:20.038 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:20.047 INFO:teuthology.orchestra.run.smithi053.stdout:920 writesame 0x9315fb0 thru 0x931f0bd data_size 0x9(0x910e bytes) 2024-09-13T05:02:20.047 INFO:teuthology.orchestra.run.smithi053.stdout:922 clone 7 order 19 su 16384 sc 7 2024-09-13T05:02:20.073 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.073 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.082 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:20.126 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:20.125+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:20.184 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.184 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.192 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:20.234 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:20.233+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:20.299 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.299 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.309 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:20.319 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:20.319+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:20.415 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.416 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.420 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:20.419+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:20.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:20.522 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:20.521+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:20.571 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.572 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.580 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:20.623 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:20.623+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:20.686 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.687 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.695 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:20.724 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:20.723+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:20.801 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.802 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.810 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:20.825 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:20.825+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:20.916 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:20.917 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:20.926 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:20.925+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:20.926 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:02:20.928 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:21.027 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:21.027+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:21.040 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:21.040 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:21.050 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:21.077 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-13T05:02:21.078 INFO:teuthology.orchestra.run.smithi053.stdout:923 writesame 0x58dff57 thru 0x58e2879 data_size 0x1(0x2923 bytes) 2024-09-13T05:02:21.100 INFO:teuthology.orchestra.run.smithi053.stdout:925 write 0xdf6aea9 thru 0xdf74c2e (0x9d86 bytes) 2024-09-13T05:02:21.106 INFO:teuthology.orchestra.run.smithi053.stdout:926 write 0x28fa15b thru 0x28ffdde (0x5c84 bytes) 2024-09-13T05:02:21.110 INFO:teuthology.orchestra.run.smithi053.stdout:927 write 0xa351c02 thru 0xa35e2f8 (0xc6f7 bytes) 2024-09-13T05:02:21.116 INFO:teuthology.orchestra.run.smithi053.stdout:928 read 0x60da54 thru 0x618037 (0xa5e4 bytes) 2024-09-13T05:02:21.116 INFO:teuthology.orchestra.run.smithi053.stdout:929 write 0xbf2d2f0 thru 0xbf3c139 (0xee4a bytes) 2024-09-13T05:02:21.126 INFO:teuthology.orchestra.run.smithi053.stdout:930 trunc from 0xdf74c2f to 0xb90c91e 2024-09-13T05:02:21.128 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:21.128+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:21.138 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.138 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.138 INFO:teuthology.orchestra.run.smithi053.stdout:936 write 0x314b91d thru 0x314d5b4 (0x1c98 bytes) 2024-09-13T05:02:21.141 INFO:teuthology.orchestra.run.smithi053.stdout:937 writesame 0x2c0297f thru 0x2c04e5b data_size 0x1(0x24dd bytes) 2024-09-13T05:02:21.147 INFO:teuthology.orchestra.run.smithi053.stdout:939 write 0x680638d thru 0x6807db8 (0x1a2c bytes) 2024-09-13T05:02:21.150 INFO:teuthology.orchestra.run.smithi053.stdout:940 read 0x766fcb6 thru 0x7673417 (0x3762 bytes) 2024-09-13T05:02:21.150 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.150 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.158 INFO:teuthology.orchestra.run.smithi053.stdout:944 writesame 0xdbb4c63 thru 0xdbbcb1a data_size 0x14(0x7eb8 bytes) 2024-09-13T05:02:21.158 INFO:teuthology.orchestra.run.smithi053.stdout:947 writesame 0x2a9dea0 thru 0x2aa6cdf data_size 0x4(0x8e40 bytes) 2024-09-13T05:02:21.164 INFO:teuthology.orchestra.run.smithi053.stdout:948 read 0x435df6f thru 0x43676d8 (0x976a bytes) 2024-09-13T05:02:21.164 INFO:teuthology.orchestra.run.smithi053.stdout:949 read 0x551f14b thru 0x552532e (0x61e4 bytes) 2024-09-13T05:02:21.164 INFO:teuthology.orchestra.run.smithi053.stdout:950 read 0x8612c1e thru 0x861d73e (0xab21 bytes) 2024-09-13T05:02:21.164 INFO:teuthology.orchestra.run.smithi053.stdout:951 writesame 0x7f240d8 thru 0x7f280e9 data_size 0x2(0x4012 bytes) 2024-09-13T05:02:21.170 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.170 INFO:teuthology.orchestra.run.smithi053.stdout:953 read 0x608f143 thru 0x6098271 (0x912f bytes) 2024-09-13T05:02:21.170 INFO:teuthology.orchestra.run.smithi053.stdout:954 write 0x710751c thru 0x7113494 (0xbf79 bytes) 2024-09-13T05:02:21.176 INFO:teuthology.orchestra.run.smithi053.stdout:955 punch from 0xb780269 to 0xb78f161, (0xeef8 bytes) 2024-09-13T05:02:21.176 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.176 INFO:teuthology.orchestra.run.smithi053.stdout:958 read 0x17fbf59 thru 0x1803623 (0x76cb bytes) 2024-09-13T05:02:21.176 INFO:teuthology.orchestra.run.smithi053.stdout:960 write 0x15f9934 thru 0x1601d59 (0x8426 bytes) 2024-09-13T05:02:21.179 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:21.179 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:964 punch from 0x8dac764 to 0x8db65d8, (0x9e74 bytes) 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:965 punch from 0x63d1ce to 0x643e50, (0x6c82 bytes) 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:966 read 0x9e3a423 thru 0x9e48ca8 (0xe886 bytes) 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:969 read 0xaa55a5e thru 0xaa63851 (0xddf4 bytes) 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:970 read 0xef47c0 thru 0xefa15d (0x599e bytes) 2024-09-13T05:02:21.181 INFO:teuthology.orchestra.run.smithi053.stdout:972 punch from 0x800a03b to 0x800ceaf, (0x2e74 bytes) 2024-09-13T05:02:21.188 INFO:teuthology.orchestra.run.smithi053.stdout:973 trunc from 0xdbbcb1b to 0x6a545af 2024-09-13T05:02:21.194 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:21.206 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.213 INFO:teuthology.orchestra.run.smithi053.stdout:976 trunc from 0x6a545af to 0xd5b7b7f 2024-09-13T05:02:21.219 INFO:teuthology.orchestra.run.smithi053.stdout:977 read 0x7d1077b thru 0x7d1153c (0xdc2 bytes) 2024-09-13T05:02:21.219 INFO:teuthology.orchestra.run.smithi053.stdout:978 writesame 0x632e258 thru 0x6333f10 data_size 0x1(0x5cb9 bytes) 2024-09-13T05:02:21.229 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:21.229+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:21.230 INFO:teuthology.orchestra.run.smithi053.stdout:980 punch from 0x6667b9 to 0x66d374, (0x6bbb bytes) 2024-09-13T05:02:21.230 INFO:teuthology.orchestra.run.smithi053.stdout:981 writesame 0x2ee67ca thru 0x2eed71b data_size 0x1(0x6f52 bytes) 2024-09-13T05:02:21.241 INFO:teuthology.orchestra.run.smithi053.stdout:983 write 0x941615e thru 0x941dbbf (0x7a62 bytes) 2024-09-13T05:02:21.245 INFO:teuthology.orchestra.run.smithi053.stdout:984 read 0xb964b97 thru 0xb974319 (0xf783 bytes) 2024-09-13T05:02:21.252 INFO:teuthology.orchestra.run.smithi053.stdout:986 write 0xe6170cc thru 0xe61e51a (0x744f bytes) 2024-09-13T05:02:21.257 INFO:teuthology.orchestra.run.smithi053.stdout:987 writesame 0xadf4ae5 thru 0xadfef16 data_size 0x1(0xa432 bytes) 2024-09-13T05:02:21.267 INFO:teuthology.orchestra.run.smithi053.stdout:988 read 0xe47775c thru 0xe486f07 (0xf7ac bytes) 2024-09-13T05:02:21.267 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:21.269 INFO:teuthology.orchestra.run.smithi053.stdout:991 trunc from 0xe61e51b to 0xcae2efa 2024-09-13T05:02:21.279 INFO:teuthology.orchestra.run.smithi053.stdout:993 writesame 0xa5e6133 thru 0xa5f2a14 data_size 0x9(0xc8e2 bytes) 2024-09-13T05:02:21.286 INFO:teuthology.orchestra.run.smithi053.stdout:996 read 0x86118c3 thru 0x861405c (0x279a bytes) 2024-09-13T05:02:21.286 INFO:teuthology.orchestra.run.smithi053.stdout:997 clone 8 order 22 su 131072 sc 14 2024-09-13T05:02:21.326 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:21.327 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:21.330 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:21.329+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:21.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:21.431 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:21.431+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:21.482 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:21.482 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:21.492 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:21.533 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:21.532+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:21.615 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:21.633 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:21.633+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:21.638 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:21.638 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:21.648 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:21.734 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:21.733+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:21.750 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:21.750 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:21.767 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:21.768 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:21.785 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:21.785 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:21.802 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:21.802 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.2892401380000003, 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:02:21.809 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:21.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:02:21.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:21.815 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.289068833, 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:02:21.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:1992 3.8 e25/19 ondisk, result = 0)", 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.2821090999999996, 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016933153999999999, 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525028+0000", 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525038+0000", 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0042e-05 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525042+0000", 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.51e-06 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541946+0000", 2024-09-13T05:02:21.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016903400999999998 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541951+0000", 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7749999999999998e-06 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541958+0000", 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9489999999999999e-06 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.825 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:1993 3.8 e25/19 ondisk, result = 0)", 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.536834+0000", 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.2703002010000004, 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0053475750000000002, 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536834+0000", 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536834+0000", 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536833+0000", 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536838+0000", 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8010000000000003e-06 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536850+0000", 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.137e-05 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.536855+0000", 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1610000000000002e-06 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542070+0000", 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0052155639999999998 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542079+0000", 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8910000000000001e-06 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542182+0000", 2024-09-13T05:02:21.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010247900000000001 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8614 3.18 3:1f08bab0:::rbd_data.117c6c62fa22.0000000000000039:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.404439+0000", 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4026952460000004, 2024-09-13T05:02:21.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003445091, 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8614 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404439+0000", 2024-09-13T05:02:21.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404439+0000", 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404437+0000", 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404445+0000", 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7540000000000006e-06 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404460+0000", 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5308999999999999e-05 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404470+0000", 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6059999999999992e-06 2024-09-13T05:02:21.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404525+0000", 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4521999999999997e-05 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.404650+0000", 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000125079 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405131+0000", 2024-09-13T05:02:21.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048183200000000002 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407424+0000", 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022921370000000001 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407797+0000", 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037335800000000001 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407845+0000", 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7549999999999997e-05 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.407884+0000", 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9619000000000001e-05 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8617 3.b 3:d2699830:::rbd_data.117c6c62fa22.000000000000003c:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:21.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.405849+0000", 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.4012851929999997, 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0059817250000000002, 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8617 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405849+0000", 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405849+0000", 2024-09-13T05:02:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405848+0000", 2024-09-13T05:02:21.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405854+0000", 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0299999999999999e-06 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405868+0000", 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3842999999999999e-05 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405876+0000", 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5510000000000002e-06 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.405935+0000", 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8984999999999999e-05 2024-09-13T05:02:21.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.406092+0000", 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000157365 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.406990+0000", 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089812699999999997 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.410855+0000", 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0038648480000000002 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.411739+0000", 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088427600000000003 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.411797+0000", 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7354e-05 2024-09-13T05:02:21.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.411831+0000", 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4258999999999998e-05 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9143 3.7 3:e609a888:::rbd_data.117c6c62fa22.0000000000000051:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:21.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8963366000000001, 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0062636530000000001, 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9143 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:21.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910796+0000", 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910804+0000", 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8979999999999996e-06 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910823+0000", 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8680000000000001e-05 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910832+0000", 2024-09-13T05:02:21.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8249999999999994e-06 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910888+0000", 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6675000000000002e-05 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.910894+0000", 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5999999999999997e-06 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915408+0000", 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00451441 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915490+0000", 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1624000000000003e-05 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915595+0000", 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105107 2024-09-13T05:02:21.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916954+0000", 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013595149999999999 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916991+0000", 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6210000000000001e-05 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917024+0000", 2024-09-13T05:02:21.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.345e-05 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917061+0000", 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7218999999999997e-05 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9146 3.2 3:478042f5:::rbd_data.117c6c62fa22.0000000000000054:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.911306+0000", 2024-09-13T05:02:21.851 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8958284920000001, 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.006498083, 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9146 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911306+0000", 2024-09-13T05:02:21.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911306+0000", 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911305+0000", 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911310+0000", 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7309999999999998e-06 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911325+0000", 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5079000000000001e-05 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911333+0000", 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3010000000000001e-06 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911379+0000", 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6767000000000001e-05 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op must wait for map", 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911385+0000", 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8719999999999999e-06 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.915997+0000", 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0046112080000000003 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916092+0000", 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5159000000000002e-05 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916221+0000", 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012972799999999999 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917405+0000", 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001184037 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917705+0000", 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029933500000000002 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917756+0000", 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1347000000000002e-05 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917804+0000", 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7707999999999998e-05 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9147 3.2 3:4074e5b1:::rbd_data.117c6c62fa22.0000000000000055:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.911375+0000", 2024-09-13T05:02:21.858 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.895759279, 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0065962599999999996, 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9147 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911375+0000", 2024-09-13T05:02:21.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911375+0000", 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911374+0000", 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911378+0000", 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1280000000000001e-06 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911389+0000", 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0353999999999999e-05 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911397+0000", 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5750000000000001e-06 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911481+0000", 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3158000000000006e-05 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting_for_map not empty", 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.911486+0000", 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3170000000000001e-06 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916525+0000", 2024-09-13T05:02:21.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0050393149999999999 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916597+0000", 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2086000000000001e-05 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.916712+0000", 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000114832 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917412+0000", 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00069963399999999998 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917899+0000", 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00048759999999999998 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917941+0000", 2024-09-13T05:02:21.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2095999999999998e-05 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.917971+0000", 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9825999999999999e-05 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9277 3.8 3:15cdb75b:::rbd_data.117c6c62fa22.0000000000000042:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.012024+0000", 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.795109981, 2024-09-13T05:02:21.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039206780000000004, 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9277 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012024+0000", 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012024+0000", 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012022+0000", 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012032+0000", 2024-09-13T05:02:21.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0207000000000001e-05 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012052+0000", 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9278000000000001e-05 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012061+0000", 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6320000000000003e-06 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012127+0000", 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5908000000000007e-05 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012221+0000", 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4189000000000002e-05 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.012692+0000", 2024-09-13T05:02:21.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047016400000000002 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015840+0000", 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0031479300000000002 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015851+0000", 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1810999999999999e-05 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015906+0000", 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4521000000000002e-05 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.015945+0000", 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9054000000000003e-05 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9884 3.b 3:d79a88f8:::rbd_data.117c6c62fa22.00000000000000a4:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.364427+0000", 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.4427073520000002, 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0051171389999999997, 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9884 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.872 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364427+0000", 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364427+0000", 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364426+0000", 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364433+0000", 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7430000000000002e-06 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364448+0000", 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4551e-05 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364456+0000", 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8599999999999993e-06 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364514+0000", 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7933999999999998e-05 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364641+0000", 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00012772 2024-09-13T05:02:21.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.365164+0000", 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00052225399999999999 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369455+0000", 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042911379999999999 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369489+0000", 2024-09-13T05:02:21.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4637000000000001e-05 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369523+0000", 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3377999999999998e-05 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369544+0000", 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1294999999999999e-05 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:9888 3.b 3:d4331ba8:::rbd_data.117c6c62fa22.00000000000000a8:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.364982+0000", 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.4421519589999998, 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047689869999999997, 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 9888 2024-09-13T05:02:21.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364982+0000", 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364982+0000", 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364981+0000", 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364987+0000", 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.22e-06 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.364997+0000", 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0587999999999999e-05 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.365003+0000", 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8919999999999997e-06 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.367485+0000", 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024818919999999999 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.367542+0000", 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6818999999999999e-05 2024-09-13T05:02:21.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.367906+0000", 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036364399999999999 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369485+0000", 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015791900000000001 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369710+0000", 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000225536 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369732+0000", 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2252e-05 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.369751+0000", 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8825999999999999e-05 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.883 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10088 3.8 3:168e662d:::rbd_data.117c6c62fa22.0000000000000005:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.977498+0000", 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.8296362950000002, 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045091519999999998, 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.884 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10088 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977498+0000", 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977498+0000", 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977497+0000", 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977502+0000", 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1690000000000002e-06 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977510+0000", 2024-09-13T05:02:21.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9299999999999992e-06 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977517+0000", 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977563+0000", 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6359000000000001e-05 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977623+0000", 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9910000000000001e-05 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977726+0000", 2024-09-13T05:02:21.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103138 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981870+0000", 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041442429999999997 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981921+0000", 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0733999999999999e-05 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981968+0000", 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7299000000000002e-05 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982007+0000", 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8655000000000002e-05 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10090 3.1e 3:7f4fda55:::rbd_data.117c6c62fa22.0000000000000007:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.977539+0000", 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.8295957540000001, 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0042471239999999997, 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10090 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977539+0000", 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977539+0000", 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977538+0000", 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977543+0000", 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977551+0000", 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7970000000000001e-06 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977557+0000", 2024-09-13T05:02:21.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0730000000000001e-06 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978856+0000", 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012989570000000001 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978932+0000", 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5902000000000006e-05 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6", 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979097+0000", 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000164943 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.980932+0000", 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018347649999999999 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981688+0000", 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075639800000000003 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981740+0000", 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1712999999999998e-05 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981786+0000", 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5833e-05 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10091 3.8 3:11495adb:::rbd_data.117c6c62fa22.0000000000000008:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.977565+0000", 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.8295691710000002, 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0045242540000000001, 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10091 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977565+0000", 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977565+0000", 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977565+0000", 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977568+0000", 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0570000000000001e-06 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977574+0000", 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977580+0000", 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3569999999999997e-06 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978954+0000", 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013737840000000001 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979044+0000", 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0810000000000006e-05 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979205+0000", 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000160268 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981913+0000", 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0027084829999999998 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982019+0000", 2024-09-13T05:02:21.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105833 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982059+0000", 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9674e-05 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982089+0000", 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0725999999999997e-05 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10092 3.8 3:17493997:::rbd_data.117c6c62fa22.0000000000000009:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.977589+0000", 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.829545135, 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005262302, 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10092 2024-09-13T05:02:21.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977589+0000", 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977589+0000", 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977589+0000", 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977592+0000", 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1379999999999999e-06 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977599+0000", 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1829999999999997e-06 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.977605+0000", 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8259999999999998e-06 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.980641+0000", 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0030358149999999999 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.980723+0000", 2024-09-13T05:02:21.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2336999999999995e-05 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.980906+0000", 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018359600000000001 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982288+0000", 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001381094 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982766+0000", 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00047796399999999999 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982818+0000", 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2266999999999999e-05 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982851+0000", 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3649000000000003e-05 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10174 3.b 3:d79a88f8:::rbd_data.117c6c62fa22.00000000000000a4:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:19.002931+0000", 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.8042028070000002, 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041039589999999999, 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10174 2024-09-13T05:02:21.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.002931+0000", 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.002931+0000", 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.002930+0000", 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.002936+0000", 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5470000000000004e-06 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.002975+0000", 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9140999999999997e-05 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.002983+0000", 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2689999999999999e-06 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003036+0000", 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2970000000000003e-05 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003110+0000", 2024-09-13T05:02:21.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3947000000000002e-05 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003273+0000", 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000163048 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.004479+0000", 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012053999999999999 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.006960+0000", 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024815319999999998 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.007001+0000", 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0389000000000002e-05 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.007035+0000", 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4792000000000002e-05 2024-09-13T05:02:21.914 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10178 3.b 3:d4331ba8:::rbd_data.117c6c62fa22.00000000000000a8:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:19.003525+0000", 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.8036095130000001, 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0041109229999999998, 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.915 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10178 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003525+0000", 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003525+0000", 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003524+0000", 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003529+0000", 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.831e-06 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003543+0000", 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3988999999999999e-05 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003551+0000", 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6960000000000005e-06 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003758+0000", 2024-09-13T05:02:21.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000207653 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.003842+0000", 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4321999999999997e-05 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.004008+0000", 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016521799999999999 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.005188+0000", 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001180162 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.007546+0000", 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023586369999999998 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.007599+0000", 2024-09-13T05:02:21.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2707999999999997e-05 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.007636+0000", 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6489000000000002e-05 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10206 3.7 3:e78c8160:::rbd_data.117c6c62fa22.0000000000000149:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:19.010740+0000", 2024-09-13T05:02:21.921 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.7963943169999999, 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003879169, 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10206 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.010740+0000", 2024-09-13T05:02:21.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.010740+0000", 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.010739+0000", 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.010745+0000", 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7610000000000004e-06 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.010758+0000", 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2893e-05 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.010766+0000", 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1049999999999995e-06 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.011488+0000", 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072175799999999997 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.011580+0000", 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2048999999999999e-05 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.011791+0000", 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00021155800000000001 2024-09-13T05:02:21.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.012544+0000", 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00075253500000000003 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.014558+0000", 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020143050000000001 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.014584+0000", 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5581000000000001e-05 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.014619+0000", 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5539e-05 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:10578 3.1e 3:7d0f2674:::rbd_object_map.117cbf4a1622:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:19.424199+0000", 2024-09-13T05:02:21.927 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.382935405, 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0037605989999999999, 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 10578 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.424199+0000", 2024-09-13T05:02:21.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.424199+0000", 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.424198+0000", 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.424202+0000", 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.179e-06 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.424209+0000", 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5449999999999998e-06 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.424214+0000", 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2310000000000004e-06 2024-09-13T05:02:21.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.426691+0000", 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024771089999999999 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.426775+0000", 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3634999999999995e-05 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6", 2024-09-13T05:02:21.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.427033+0000", 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00025818199999999998 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.427712+0000", 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067883799999999997 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.427902+0000", 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00019024100000000001 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.427931+0000", 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8231000000000001e-05 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:19.427959+0000", 2024-09-13T05:02:21.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8894000000000002e-05 2024-09-13T05:02:21.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:21.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:21.936 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:21.937 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:21.937+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:21.938 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:21.939 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:21.939 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:21.939 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:21.939 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:21.940 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:21.940 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:21.940 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:21.941 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:21.982 INFO:teuthology.orchestra.run.smithi053.stdout:flattening image image_client.0-clone7 2024-09-13T05:02:22.039 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:22.038+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:22.047 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:22.047 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:22.056 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:22.128 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-13T05:02:22.139 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:22.139+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:22.202 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:22.202 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:22.202 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:22.202 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:22.202 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:22.203 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:22.204 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:22.204 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T05:02:22.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T05:02:22.210 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:22.252 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:22.251+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:22.303 INFO:teuthology.orchestra.run.smithi053.stdout:998 write 0xdd7fa24 thru 0xdd80125 (0x702 bytes) 2024-09-13T05:02:22.308 INFO:teuthology.orchestra.run.smithi053.stdout:999 punch from 0xcd1593f to 0xcd19f84, (0x4645 bytes) 2024-09-13T05:02:22.308 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.308 INFO:teuthology.orchestra.run.smithi053.stdout:1001 read 0xdacbb6e thru 0xdad55ed (0x9a80 bytes) 2024-09-13T05:02:22.308 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.308 INFO:teuthology.orchestra.run.smithi053.stdout:1003 write 0x1bdc77b thru 0x1beb615 (0xee9b bytes) 2024-09-13T05:02:22.313 INFO:teuthology.orchestra.run.smithi053.stdout:1004 writesame 0xbf772e1 thru 0xbf83338 data_size 0x14(0xc058 bytes) 2024-09-13T05:02:22.319 INFO:teuthology.orchestra.run.smithi053.stdout:1005 read 0x35e9f6d thru 0x35ed216 (0x32aa bytes) 2024-09-13T05:02:22.320 INFO:teuthology.orchestra.run.smithi053.stdout:1006 write 0xc68ed08 thru 0xc68f140 (0x439 bytes) 2024-09-13T05:02:22.321 INFO:teuthology.orchestra.run.smithi053.stdout:1007 writesame 0x9b2789f thru 0x9b2bcec data_size 0x7(0x444e bytes) 2024-09-13T05:02:22.326 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.326 INFO:teuthology.orchestra.run.smithi053.stdout:1011 write 0xace030f thru 0xace670f (0x6401 bytes) 2024-09-13T05:02:22.330 INFO:teuthology.orchestra.run.smithi053.stdout:1012 punch from 0x89fb485 to 0x89fc02b, (0xba6 bytes) 2024-09-13T05:02:22.330 INFO:teuthology.orchestra.run.smithi053.stdout:1013 write 0x368d0fc thru 0x36979d4 (0xa8d9 bytes) 2024-09-13T05:02:22.337 INFO:teuthology.orchestra.run.smithi053.stdout:1014 punch from 0x4ddf2ae to 0x4de9d89, (0xaadb bytes) 2024-09-13T05:02:22.337 INFO:teuthology.orchestra.run.smithi053.stdout:1015 read 0x6fa347f thru 0x6fab155 (0x7cd7 bytes) 2024-09-13T05:02:22.337 INFO:teuthology.orchestra.run.smithi053.stdout:1016 read 0x4874dd3 thru 0x4880cce (0xbefc bytes) 2024-09-13T05:02:22.337 INFO:teuthology.orchestra.run.smithi053.stdout:1018 write 0x2b6cc9b thru 0x2b761c6 (0x952c bytes) 2024-09-13T05:02:22.341 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:22.341+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:22.342 INFO:teuthology.orchestra.run.smithi053.stdout:1019 punch from 0xcbfcc1f to 0xcc01624, (0x4a05 bytes) 2024-09-13T05:02:22.343 INFO:teuthology.orchestra.run.smithi053.stdout:1020 read 0x1ed6beb thru 0x1ee322e (0xc644 bytes) 2024-09-13T05:02:22.343 INFO:teuthology.orchestra.run.smithi053.stdout:1021 write 0xd1413b3 thru 0xd14f264 (0xdeb2 bytes) 2024-09-13T05:02:22.345 INFO:teuthology.orchestra.run.smithi053.stdout:1022 read 0xd0a7151 thru 0xd0b5d4a (0xebfa bytes) 2024-09-13T05:02:22.345 INFO:teuthology.orchestra.run.smithi053.stdout:1024 writesame 0x533417a thru 0x533a253 data_size 0x62(0x60da bytes) 2024-09-13T05:02:22.349 INFO:teuthology.orchestra.run.smithi053.stdout:1025 punch from 0x91e3bff to 0x91ec934, (0x8d35 bytes) 2024-09-13T05:02:22.357 INFO:teuthology.orchestra.run.smithi053.stdout:1026 trunc from 0xdd80126 to 0x6bbdff6 2024-09-13T05:02:22.368 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:22.368 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8559 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.9933356409999998, 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011244857, 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.369 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8559 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371554+0000", 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371562+0000", 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5789999999999998e-06 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371569+0000", 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4209999999999998e-06 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371574+0000", 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6970000000000001e-06 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.380413+0000", 2024-09-13T05:02:22.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0088391589999999992 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.380468+0000", 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5303000000000002e-05 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.380819+0000", 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035133799999999998 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382747+0000", 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019273579999999999 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382751+0000", 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382780+0000", 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8225000000000002e-05 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382799+0000", 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9607e-05 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8560 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.9933119999999995, 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011621819, 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.375 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8560 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371578+0000", 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371580+0000", 2024-09-13T05:02:22.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371586+0000", 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.959e-06 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.371596+0000", 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2609999999999999e-06 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.381495+0000", 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098995750000000007 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.381578+0000", 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2659999999999998e-05 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.381817+0000", 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00023948 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.382805+0000", 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00098802400000000011 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.383153+0000", 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034762200000000001 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.383180+0000", 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6961000000000002e-05 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.383200+0000", 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9928e-05 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8608 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.9683735049999997, 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012637699000000001, 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.382 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8608 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:22.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396516+0000", 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396518+0000", 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9139999999999998e-06 2024-09-13T05:02:22.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396521+0000", 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9780000000000001e-06 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.396523+0000", 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2010000000000002e-06 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.406426+0000", 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0099030820000000006 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.406476+0000", 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9172000000000003e-05 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.406741+0000", 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00026489699999999999 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.407340+0000", 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00059977299999999995 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409063+0000", 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017231810000000001 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409111+0000", 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7657000000000001e-05 2024-09-13T05:02:22.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409154+0000", 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3059999999999998e-05 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9464 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.124251+0000", 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.2406394790000004, 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011271794999999999, 2024-09-13T05:02:22.388 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9464 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124251+0000", 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124251+0000", 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124250+0000", 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124252+0000", 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6390000000000001e-06 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124255+0000", 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.027e-06 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.124257+0000", 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2689999999999998e-06 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134051+0000", 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.009794054 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134107+0000", 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5452999999999999e-05 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134303+0000", 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000196248 2024-09-13T05:02:22.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.134921+0000", 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061787599999999995 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135482+0000", 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00056160499999999996 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135505+0000", 2024-09-13T05:02:22.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3025000000000001e-05 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.135522+0000", 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6824e-05 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9699 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.1171647739999999, 2024-09-13T05:02:22.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011145735, 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9699 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247725+0000", 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247728+0000", 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6450000000000001e-06 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247733+0000", 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8440000000000003e-06 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247738+0000", 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000003e-06 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257135+0000", 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0093971109999999997 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257187+0000", 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2012000000000003e-05 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257436+0000", 2024-09-13T05:02:22.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024863400000000002 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258821+0000", 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013847659999999999 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258826+0000", 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4319999999999998e-06 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258854+0000", 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7676999999999999e-05 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258871+0000", 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7292e-05 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9700 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.1171478419999996, 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013001805, 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9700 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247742+0000", 2024-09-13T05:02:22.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247745+0000", 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.943e-06 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247751+0000", 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247755+0000", 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5530000000000003e-06 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257845+0000", 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010090238 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.257894+0000", 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8194e-05 2024-09-13T05:02:22.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258138+0000", 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000244305 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260659+0000", 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025215709999999998 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260690+0000", 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0670999999999998e-05 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260716+0000", 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6046000000000001e-05 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260744+0000", 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.781e-05 2024-09-13T05:02:22.406 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9701 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.117130586, 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013027680999999999, 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.407 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9701 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247759+0000", 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247762+0000", 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9119999999999998e-06 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247768+0000", 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1530000000000002e-06 2024-09-13T05:02:22.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247773+0000", 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5870000000000003e-06 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258914+0000", 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011140957 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.258992+0000", 2024-09-13T05:02:22.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8226999999999995e-05 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.259350+0000", 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000358625 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260686+0000", 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013354809999999999 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260750+0000", 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3718000000000002e-05 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260771+0000", 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1379999999999999e-05 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260787+0000", 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6081000000000001e-05 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.412 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9702 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.247777+0000", 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.1171133470000001, 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.013366486, 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.413 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9702 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247777+0000", 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247777+0000", 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247776+0000", 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247779+0000", 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5959999999999998e-06 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247785+0000", 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8690000000000001e-06 2024-09-13T05:02:22.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.247789+0000", 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3789999999999999e-06 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.259882+0000", 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012092601 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.259958+0000", 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6709e-05 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260278+0000", 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031917799999999999 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.260895+0000", 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061692099999999998 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.261098+0000", 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000203527 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.261125+0000", 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6667000000000001e-05 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.261143+0000", 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8383999999999999e-05 2024-09-13T05:02:22.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9847 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0283486909999997, 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011087779000000001, 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9847 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:22.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336541+0000", 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336544+0000", 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9689999999999999e-06 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336550+0000", 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7910000000000001e-06 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.336554+0000", 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5669999999999996e-06 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.345827+0000", 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0092730469999999995 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.345909+0000", 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2062000000000001e-05 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.346211+0000", 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030132100000000001 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347552+0000", 2024-09-13T05:02:22.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001341334 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347572+0000", 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0239999999999999e-05 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347598+0000", 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5647999999999999e-05 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.347629+0000", 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1136999999999997e-05 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9872 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0126165890000003, 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012063077, 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.425 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9872 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352273+0000", 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352275+0000", 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5510000000000001e-06 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352278+0000", 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0690000000000002e-06 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352280+0000", 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4590000000000001e-06 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.358767+0000", 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064866180000000004 2024-09-13T05:02:22.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.358856+0000", 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9518000000000006e-05 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.359212+0000", 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035534599999999998 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.361913+0000", 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027010279999999998 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364249+0000", 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023360870000000001 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364296+0000", 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7497999999999998e-05 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364336+0000", 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0105000000000003e-05 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9873 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0126076590000004, 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012174516999999999, 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9873 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352282+0000", 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352284+0000", 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.575e-06 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352287+0000", 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0019999999999998e-06 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352289+0000", 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.48e-06 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360058+0000", 2024-09-13T05:02:22.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0077688089999999998 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360131+0000", 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3201000000000006e-05 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360323+0000", 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019145499999999999 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.361938+0000", 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001615294 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364347+0000", 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024086429999999998 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364406+0000", 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9490000000000001e-05 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364457+0000", 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.079e-05 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9874 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0125931750000001, 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012309581999999999, 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9874 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352297+0000", 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352298+0000", 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8700000000000001e-06 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352305+0000", 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1940000000000003e-06 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352307+0000", 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.813e-06 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360786+0000", 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084785250000000006 2024-09-13T05:02:22.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360876+0000", 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0301000000000002e-05 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.361302+0000", 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00042560599999999998 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364513+0000", 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0032107390000000002 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364555+0000", 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2234000000000003e-05 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364585+0000", 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9833e-05 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364606+0000", 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1701000000000001e-05 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9875 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0125801550000002, 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012340808, 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9875 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352310+0000", 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352311+0000", 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6309999999999999e-06 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352315+0000", 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352317+0000", 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.616e-06 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362194+0000", 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098765569999999994 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362291+0000", 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7231999999999993e-05 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362637+0000", 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034592500000000002 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364549+0000", 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019119149999999999 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364612+0000", 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3373000000000005e-05 2024-09-13T05:02:22.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364635+0000", 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2214000000000001e-05 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364651+0000", 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6137000000000002e-05 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9876 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.0125685449999997, 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012573533, 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9876 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352321+0000", 2024-09-13T05:02:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352323+0000", 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7719999999999999e-06 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352330+0000", 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1829999999999997e-06 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.352333+0000", 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8380000000000002e-06 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.363216+0000", 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010882724 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.363315+0000", 2024-09-13T05:02:22.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8782000000000004e-05 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.363710+0000", 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00039558399999999999 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364660+0000", 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00094961899999999996 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364839+0000", 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000179109 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364867+0000", 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8082999999999999e-05 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364895+0000", 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8070000000000001e-05 2024-09-13T05:02:22.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9898 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9946130520000001, 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011213851, 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9898 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370277+0000", 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370278+0000", 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.652e-06 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370282+0000", 2024-09-13T05:02:22.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3809999999999999e-06 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370284+0000", 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3860000000000001e-06 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.378618+0000", 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0083340589999999996 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.378698+0000", 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9726000000000002e-05 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379083+0000", 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038519800000000001 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380487+0000", 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014035409999999999 2024-09-13T05:02:22.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381412+0000", 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00092494699999999997 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381456+0000", 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3918000000000002e-05 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381491+0000", 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5244999999999999e-05 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9899 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370288+0000", 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9946023500000001, 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012050751, 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9899 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370288+0000", 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370288+0000", 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370287+0000", 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370289+0000", 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7409999999999999e-06 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370294+0000", 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8949999999999997e-06 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370297+0000", 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5840000000000002e-06 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379565+0000", 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0092685560000000007 2024-09-13T05:02:22.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379648+0000", 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3164999999999994e-05 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.379948+0000", 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029975599999999998 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382279+0000", 2024-09-13T05:02:22.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023305890000000001 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382284+0000", 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.254e-06 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382316+0000", 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2004000000000001e-05 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382338+0000", 2024-09-13T05:02:22.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2449000000000001e-05 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9900 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9945818920000002, 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012108782, 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.468 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9900 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370308+0000", 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370310+0000", 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3350000000000001e-06 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370315+0000", 2024-09-13T05:02:22.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7339999999999999e-06 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370319+0000", 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2209999999999999e-06 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380521+0000", 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010201511 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380602+0000", 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1211000000000006e-05 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.380959+0000", 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035688000000000001 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382362+0000", 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014032389999999999 2024-09-13T05:02:22.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382369+0000", 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3220000000000004e-06 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382400+0000", 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0814e-05 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382417+0000", 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6835000000000001e-05 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9901 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9945681730000002, 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012481054, 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.474 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9901 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370322+0000", 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370323+0000", 2024-09-13T05:02:22.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6649999999999999e-06 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370326+0000", 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.370329+0000", 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5330000000000001e-06 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381540+0000", 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011211087 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381609+0000", 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8922000000000002e-05 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.381838+0000", 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00022881999999999999 2024-09-13T05:02:22.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382611+0000", 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00077353600000000002 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382744+0000", 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013245100000000001 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382772+0000", 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.798e-05 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.382803+0000", 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1207000000000001e-05 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9982 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9583766049999998, 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011655957, 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9982 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406513+0000", 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406516+0000", 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2490000000000001e-06 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406522+0000", 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0240000000000003e-06 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406527+0000", 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5009999999999998e-06 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.415355+0000", 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0088282719999999999 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.415430+0000", 2024-09-13T05:02:22.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4474999999999994e-05 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.415774+0000", 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034416900000000002 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418070+0000", 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022960599999999999 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418105+0000", 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5194999999999998e-05 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418134+0000", 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8878999999999999e-05 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418169+0000", 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5502e-05 2024-09-13T05:02:22.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9983 3.1a 3:58c7e992:::rbd_object_map.117c6c62fa22:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.958356674, 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.011711048, 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.487 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9983 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406533+0000", 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406537+0000", 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8550000000000004e-06 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406543+0000", 2024-09-13T05:02:22.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.127e-06 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.406548+0000", 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9030000000000003e-06 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.416447+0000", 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0098995809999999993 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.416543+0000", 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5599000000000004e-05 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.416861+0000", 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00031824199999999999 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418196+0000", 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013346230000000001 2024-09-13T05:02:22.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418200+0000", 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8340000000000001e-06 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418225+0000", 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5119000000000001e-05 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.418244+0000", 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9556999999999999e-05 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.493 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:22.495 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:22.497 INFO:teuthology.orchestra.run.smithi053.stdout:1027 writesame 0x8f3dc5f thru 0x8f4663c data_size 0x7(0x89de bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1028 write 0x45e3f81 thru 0x45e687c (0x28fc bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1029 read 0x4072155 thru 0x4080a00 (0xe8ac bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1030 read 0x59845d8 thru 0x59861ce (0x1bf7 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1032 trunc from 0x8f4663d to 0x7c7307a 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1034 read 0x1c059b thru 0x1c0805 (0x26b bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1035 writesame 0x40f5256 thru 0x41041cb data_size 0x11(0xef76 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1037 read 0x178b29a thru 0x179ab82 (0xf8e9 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1039 read 0x491b8b2 thru 0x491b916 (0x65 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1040 trunc from 0x7c7307a to 0x7537141 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1041 punch from 0x7301dd to 0x7323a4, (0x21c7 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1042 read 0x42a795d thru 0x42b0825 (0x8ec9 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1043 read 0x30ac939 thru 0x30b7f6f (0xb637 bytes) 2024-09-13T05:02:22.498 INFO:teuthology.orchestra.run.smithi053.stdout:1044 read 0x5bbea91 thru 0x5bc6119 (0x7689 bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1046 punch from 0x63061bb to 0x6312fe3, (0xce28 bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1048 read 0x11f1630 thru 0x11ff5d1 (0xdfa2 bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1049 trunc from 0x7537141 to 0x1161cab 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1054 write 0x1116b39 thru 0x111addd (0x42a5 bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1055 write 0xae8d71 thru 0xae9def (0x107f bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1056 trunc from 0x1161cab to 0xe2590d7 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1059 punch from 0x5cd99d8 to 0x5ce1eea, (0x8512 bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1060 read 0x42bed00 thru 0x42c127e (0x257f bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1061 read 0x7cff6f8 thru 0x7d0bc04 (0xc50d bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1062 write 0xa1958c4 thru 0xa1a1ce8 (0xc425 bytes) 2024-09-13T05:02:22.499 INFO:teuthology.orchestra.run.smithi053.stdout:1063 read 0xac5aee0 thru 0xac63af6 (0x8c17 bytes) 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:1067 read 0xa583789 thru 0xa585922 (0x219a bytes) 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:1069 write 0xea16a68 thru 0xea1b041 (0x45da bytes) 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:1070 read 0x553326a thru 0x5539a44 (0x67db bytes) 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:1071 write 0xcc76f8d thru 0xcc81539 (0xa5ad bytes) 2024-09-13T05:02:22.500 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:22.494+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:22.500 INFO:teuthology.orchestra.run.smithi053.stdout:1072 punch from 0xaed006 to 0xaf52fc, (0x82f6 bytes) 2024-09-13T05:02:22.502 INFO:teuthology.orchestra.run.smithi053.stdout:1074 write 0xee0b65b thru 0xee1b01c (0xf9c2 bytes) 2024-09-13T05:02:22.505 INFO:teuthology.orchestra.run.smithi053.stdout:1075 read 0x27b34a4 thru 0x27b8a63 (0x55c0 bytes) 2024-09-13T05:02:22.505 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:22.520 INFO:teuthology.orchestra.run.smithi053.stdout:1077 trunc from 0xee1b01d to 0x1ef1b8a 2024-09-13T05:02:22.533 INFO:teuthology.orchestra.run.smithi053.stdout:1078 clone 9 order 22 su 262144 sc 2 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 31, 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "01e1476b-1467-4e0e-b01c-0f8cdf8276fe", 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-13T05:01:45.075461+0000", 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-13T05:02:21.922767+0000", 2024-09-13T05:02:22.535 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-13T05:01:55.160352+0000", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-13T05:01:47.179750+0000", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5806080, 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 4, 2024-09-13T05:02:22.536 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "reef", 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-13T05:02:22.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.817858+0000", 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:22.538 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.539 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "17", 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 3, 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 17, 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:22.540 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.541 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:22.542 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.543 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.981425+0000", 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:22.544 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:22.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "16", 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:22.546 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.547 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-13T05:02:22.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 8, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 8, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.25, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:02:05.508713+0000", 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:22.550 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:22.551 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "31", 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 11, 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 31, 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:22.552 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:22.553 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:22.554 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.5, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.5, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.5, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.5, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:22.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "baf16a5c-4204-424b-8d8e-a5b1486aa052", 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.556 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-13T05:02:22.557 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-13T05:02:22.558 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.561 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-13T05:02:22.562 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1769628429", 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1769628429", 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1769628429", 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1769628429", 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f3df9a0f-3a9e-4665-9233-718bc0fe1223", 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.564 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.566 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.567 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-13T05:02:22.568 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/827277153", 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/827277153", 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/827277153", 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/827277153", 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1304a876-75c5-454b-a60b-8c9823c4aca8", 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 0, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.570 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.571 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/1334123504", 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/1334123504", 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/1334123504", 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/1334123504", 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:02:22.575 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "83668012-8ddb-4fe5-bb31-6e78d2723f12", 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.576 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6824", 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6825", 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6826", 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6827", 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6830", 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6831", 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6828", 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6829", 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6825/4134389313", 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6827/4134389313", 2024-09-13T05:02:22.580 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6831/4134389313", 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6829/4134389313", 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f462232f-5144-4652-9139-9039d53e191a", 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.581 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6808", 2024-09-13T05:02:22.582 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6809", 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6810", 2024-09-13T05:02:22.583 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6811", 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6815", 2024-09-13T05:02:22.584 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6816", 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.585 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6812", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6813", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6809/3681797816", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6811/3681797816", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6816/3681797816", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6813/3681797816", 2024-09-13T05:02:22.586 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1d2d8607-f1d9-45e3-8486-3f5bb23b516b", 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.587 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6800", 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.588 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6801", 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6802", 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.589 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6803", 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6806", 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.590 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6807", 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6804", 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.591 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6805", 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6801/81482652", 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6803/81482652", 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6807/81482652", 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6805/81482652", 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "7fe0fb7d-0a40-4666-82a1-4e18f53b445c", 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.593 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6814", 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6817", 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.594 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6818", 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6819", 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.595 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6822", 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6823", 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6820", 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6821", 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6817/576600432", 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6819/576600432", 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6823/576600432", 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6821/576600432", 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.598 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "811503be-abe3-4d64-a094-09ff675f3666", 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:22.599 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6824", 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.600 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6825", 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6826", 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.601 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6827", 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6830", 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.602 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6831", 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6828", 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6829", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6825/896779674", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6827/896779674", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6831/896779674", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6829/896779674", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:22.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358655+0000", 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.605 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.343203+0000", 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.606 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.307081+0000", 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.607 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.315153+0000", 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.280902+0000", 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.302500+0000", 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.609 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.354685+0000", 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:22.610 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358223+0000", 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-13T05:02:22.611 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-13T05:02:22.612 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-13T05:02:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-13T05:02:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-13T05:02:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-13T05:02:22.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:22.613 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:22.615 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:22.616 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:22.616 INFO:tasks.thrashosds.thrasher:j is {'epoch': 31, 'fsid': '01e1476b-1467-4e0e-b01c-0f8cdf8276fe', 'created': '2024-09-13T05:01:45.075461+0000', 'modified': '2024-09-13T05:02:21.922767+0000', 'last_up_change': '2024-09-13T05:01:55.160352+0000', 'last_in_change': '2024-09-13T05:01:47.179750+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T05:01:55.817858+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T05:01:55.981425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T05:02:05.508713+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '31', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 11, 'snap_epoch': 31, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'baf16a5c-4204-424b-8d8e-a5b1486aa052', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1769628429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1769628429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1769628429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1769628429}]}, 'public_addr': '172.21.15.53:6817/1769628429', 'cluster_addr': '172.21.15.53:6819/1769628429', 'heartbeat_back_addr': '172.21.15.53:6823/1769628429', 'heartbeat_front_addr': '172.21.15.53:6821/1769628429', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f3df9a0f-3a9e-4665-9233-718bc0fe1223', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 827277153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 827277153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 827277153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 827277153}]}, 'public_addr': '172.21.15.53:6801/827277153', 'cluster_addr': '172.21.15.53:6803/827277153', 'heartbeat_back_addr': '172.21.15.53:6807/827277153', 'heartbeat_front_addr': '172.21.15.53:6805/827277153', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1304a876-75c5-454b-a60b-8c9823c4aca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 1334123504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 1334123504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 1334123504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 1334123504}]}, 'public_addr': '172.21.15.53:6809/1334123504', 'cluster_addr': '172.21.15.53:6811/1334123504', 'heartbeat_back_addr': '172.21.15.53:6815/1334123504', 'heartbeat_front_addr': '172.21.15.53:6813/1334123504', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '83668012-8ddb-4fe5-bb31-6e78d2723f12', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6824', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6825', 'nonce': 4134389313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6826', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6827', 'nonce': 4134389313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6830', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6831', 'nonce': 4134389313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6828', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6829', 'nonce': 4134389313}]}, 'public_addr': '172.21.15.53:6825/4134389313', 'cluster_addr': '172.21.15.53:6827/4134389313', 'heartbeat_back_addr': '172.21.15.53:6831/4134389313', 'heartbeat_front_addr': '172.21.15.53:6829/4134389313', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f462232f-5144-4652-9139-9039d53e191a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 3681797816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 3681797816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6815', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6816', 'nonce': 3681797816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 3681797816}]}, 'public_addr': '172.21.15.110:6809/3681797816', 'cluster_addr': '172.21.15.110:6811/3681797816', 'heartbeat_back_addr': '172.21.15.110:6816/3681797816', 'heartbeat_front_addr': '172.21.15.110:6813/3681797816', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1d2d8607-f1d9-45e3-8486-3f5bb23b516b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 81482652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 81482652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 81482652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 81482652}]}, 'public_addr': '172.21.15.110:6801/81482652', 'cluster_addr': '172.21.15.110:6803/81482652', 'heartbeat_back_addr': '172.21.15.110:6807/81482652', 'heartbeat_front_addr': '172.21.15.110:6805/81482652', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7fe0fb7d-0a40-4666-82a1-4e18f53b445c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 576600432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 576600432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 576600432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 576600432}]}, 'public_addr': '172.21.15.110:6817/576600432', 'cluster_addr': '172.21.15.110:6819/576600432', 'heartbeat_back_addr': '172.21.15.110:6823/576600432', 'heartbeat_front_addr': '172.21.15.110:6821/576600432', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '811503be-abe3-4d64-a094-09ff675f3666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6824', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6825', 'nonce': 896779674}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6826', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6827', 'nonce': 896779674}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6830', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6831', 'nonce': 896779674}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6828', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6829', 'nonce': 896779674}]}, 'public_addr': '172.21.15.110:6825/896779674', 'cluster_addr': '172.21.15.110:6827/896779674', 'heartbeat_back_addr': '172.21.15.110:6831/896779674', 'heartbeat_front_addr': '172.21.15.110:6829/896779674', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358655+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.343203+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.307081+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.315153+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.280902+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.302500+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.354685+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T05:02:22.617 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:02:22.617 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:22.618 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:22.613+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:22.715 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:22.714+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:22.757 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:22.760 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:22.760 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:22.760 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:22.760 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:22.766 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:22.815 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:22.815+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:22.911 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.024044151, 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002690039, 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.912 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886006+0000", 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886013+0000", 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886021+0000", 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3070000000000003e-06 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886030+0000", 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4909999999999994e-06 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886071+0000", 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886076+0000", 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1679999999999997e-06 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887440+0000", 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001364142 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887556+0000", 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011576799999999999 2024-09-13T05:02:22.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887726+0000", 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017057500000000001 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888284+0000", 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00055794299999999998 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888637+0000", 2024-09-13T05:02:22.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035292599999999999 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888674+0000", 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7416000000000002e-05 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888698+0000", 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3422999999999999e-05 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:1936 3.1c 3:3ac27fc6:::rbd_data.117cdd702390.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.4660832940000006, 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065407260000000002, 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1936 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.919 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443968+0000", 2024-09-13T05:02:22.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443973+0000", 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1240000000000004e-06 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443984+0000", 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0796e-05 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443990+0000", 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.939e-06 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.444026+0000", 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5697000000000001e-05 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.444119+0000", 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2684999999999998e-05 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.445186+0000", 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001067274 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450407+0000", 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052206129999999998 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450454+0000", 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7132999999999997e-05 2024-09-13T05:02:22.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450487+0000", 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2697999999999998e-05 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450509+0000", 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2767000000000001e-05 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:2050 3.16 3:6bc3cb69:::rbd_data.117cdd702390.000000000000005a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.3326234079999999, 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036278289999999999, 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2050 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577427+0000", 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:22.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577435+0000", 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0120000000000005e-06 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577448+0000", 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577456+0000", 2024-09-13T05:02:22.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9619999999999993e-06 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577506+0000", 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9738999999999999e-05 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577611+0000", 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105074 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:22.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.578226+0000", 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061466299999999999 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.580639+0000", 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024128550000000002 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.580966+0000", 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032758400000000001 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.581016+0000", 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0105000000000002e-05 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.581056+0000", 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9962000000000001e-05 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9140 3.16 3:6bbed544:::rbd_data.117c6c62fa22.000000000000004e:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.930 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.9993214569999997, 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039799379999999997, 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9140 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:22.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910729+0000", 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910735+0000", 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2469999999999999e-06 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910746+0000", 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1048e-05 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910751+0000", 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2229999999999998e-06 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910792+0000", 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1136000000000001e-05 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6119999999999998e-06 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913554+0000", 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00275632 2024-09-13T05:02:22.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913600+0000", 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6162000000000003e-05 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913697+0000", 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6904999999999995e-05 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914317+0000", 2024-09-13T05:02:22.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062001999999999999 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914626+0000", 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030938899999999997 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914674+0000", 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7221000000000001e-05 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914710+0000", 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6718999999999999e-05 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9239 3.1d 3:ba6bf161:::rbd_data.117c6c62fa22.0000000000000151:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.9286947330000004, 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027959809999999999, 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.937 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9239 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981356+0000", 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981362+0000", 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7030000000000003e-06 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981373+0000", 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1063000000000001e-05 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981381+0000", 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2700000000000004e-06 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981418+0000", 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7049999999999999e-05 2024-09-13T05:02:22.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981500+0000", 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1798999999999994e-05 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.982080+0000", 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058020599999999999 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.983823+0000", 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017424020000000001 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984107+0000", 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028401700000000002 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984134+0000", 2024-09-13T05:02:22.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7489000000000001e-05 2024-09-13T05:02:22.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984153+0000", 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9123999999999999e-05 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9278 3.1c 3:3c83d342:::rbd_data.117c6c62fa22.0000000000000043:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.8979954719999999, 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030035639999999998, 2024-09-13T05:02:22.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9278 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012055+0000", 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012062+0000", 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6959999999999999e-06 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012076+0000", 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3994e-05 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012081+0000", 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2499999999999997e-06 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012125+0000", 2024-09-13T05:02:22.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708000000000002e-05 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012208+0000", 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2834e-05 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012661+0000", 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045269800000000002 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014746+0000", 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020852079999999999 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014992+0000", 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024621199999999999 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.015029+0000", 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6698e-05 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.015060+0000", 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1265e-05 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9394 3.1c 3:384f0210:::rbd_data.117c6c62fa22.00000000000000d5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.8185868110000003, 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002679439, 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.950 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9394 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091464+0000", 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091471+0000", 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6490000000000002e-06 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091487+0000", 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6529999999999999e-05 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091495+0000", 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2700000000000004e-06 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091552+0000", 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6740000000000002e-05 2024-09-13T05:02:22.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091635+0000", 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2509000000000006e-05 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091739+0000", 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103915 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.092622+0000", 2024-09-13T05:02:22.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00088372799999999997 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094066+0000", 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001444047 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094109+0000", 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2416999999999997e-05 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094145+0000", 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5819999999999999e-05 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9468 3.c 3:37c3b698:::rbd_data.117c6c62fa22.0000000000000162:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.781543095, 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030936710000000001, 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.956 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9468 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128507+0000", 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128515+0000", 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4789999999999996e-06 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128533+0000", 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7227999999999998e-05 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128541+0000", 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128591+0000", 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9756999999999998e-05 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128715+0000", 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123979 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.129064+0000", 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034899799999999999 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131179+0000", 2024-09-13T05:02:22.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002115268 2024-09-13T05:02:22.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131516+0000", 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033735900000000002 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131564+0000", 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7469999999999998e-05 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131602+0000", 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8892e-05 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9879 3.1d 3:b8dde86d:::rbd_data.117c6c62fa22.000000000000009e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:22.965 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5532673529999999, 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034236219999999999, 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9879 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:22.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356783+0000", 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356793+0000", 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0233e-05 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356815+0000", 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2005999999999999e-05 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356825+0000", 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7540000000000001e-06 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356892+0000", 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7088999999999994e-05 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.357053+0000", 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016157300000000001 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.357490+0000", 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043711299999999999 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360070+0000", 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025790990000000001 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360143+0000", 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3672999999999997e-05 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360175+0000", 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1690999999999999e-05 2024-09-13T05:02:22.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360208+0000", 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3260999999999999e-05 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10086 3.1c 3:385e9163:::rbd_data.117c6c62fa22.0000000000000003:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9325607959999997, 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027722110000000001, 2024-09-13T05:02:22.971 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10086 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977490+0000", 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977496+0000", 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6699999999999997e-06 2024-09-13T05:02:22.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977510+0000", 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4073000000000001e-05 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977518+0000", 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5270000000000001e-06 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977573+0000", 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5096000000000002e-05 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977663+0000", 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0365000000000006e-05 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977847+0000", 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018395400000000001 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.979691+0000", 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018432190000000001 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980183+0000", 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049266399999999997 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980228+0000", 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4202000000000001e-05 2024-09-13T05:02:22.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980263+0000", 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5822999999999999e-05 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10126 3.1c 3:3c309cef:::rbd_data.117c6c62fa22.000000000000002e:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9195525529999999, 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028308270000000002, 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.977 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10126 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990506+0000", 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8189999999999992e-06 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990516+0000", 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0033e-05 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990523+0000", 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4670000000000001e-06 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990568+0000", 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4749000000000001e-05 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990650+0000", 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2464000000000002e-05 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990806+0000", 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015617499999999999 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991958+0000", 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00115196 2024-09-13T05:02:22.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993253+0000", 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012944020000000001 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993296+0000", 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2957999999999998e-05 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993330+0000", 2024-09-13T05:02:22.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4650000000000002e-05 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10130 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9187082869999998, 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028473510000000001, 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.983 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10130 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991343+0000", 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991348+0000", 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8300000000000001e-06 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991361+0000", 2024-09-13T05:02:22.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2405000000000001e-05 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991373+0000", 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1815e-05 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991419+0000", 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6575e-05 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991592+0000", 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172972 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991755+0000", 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162318 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992830+0000", 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010755809999999999 2024-09-13T05:02:22.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994112+0000", 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012814409999999999 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994156+0000", 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3955000000000002e-05 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994191+0000", 2024-09-13T05:02:22.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5475000000000002e-05 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10187 3.c 3:356439cd:::rbd_data.117c6c62fa22.00000000000000b2:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.9035465929999997, 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025044160000000002, 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.989 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10187 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006504+0000", 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006510+0000", 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5500000000000002e-06 2024-09-13T05:02:22.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006519+0000", 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3619999999999995e-06 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006526+0000", 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6279999999999999e-06 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006577+0000", 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1347999999999997e-05 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006662+0000", 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5271000000000004e-05 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006835+0000", 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173148 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007618+0000", 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00078252200000000001 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008945+0000", 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001326767 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008976+0000", 2024-09-13T05:02:22.994 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1466e-05 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009010+0000", 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3590999999999997e-05 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10773 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 319488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3710124559999999, 2024-09-13T05:02:22.995 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026201689999999999, 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10773 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.996 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539041+0000", 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.199e-06 2024-09-13T05:02:22.997 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539047+0000", 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.57e-06 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539051+0000", 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6360000000000002e-06 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541497+0000", 2024-09-13T05:02:22.998 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024458610000000001 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541571+0000", 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4208999999999994e-05 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541660+0000", 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8208000000000007e-05 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:22.999 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10774 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.370996603, 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027446580000000001, 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.000 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10774 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.001 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539060+0000", 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8080000000000004e-06 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539066+0000", 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.784e-06 2024-09-13T05:02:23.002 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539070+0000", 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7949999999999998e-06 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541689+0000", 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026189709999999999 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541743+0000", 2024-09-13T05:02:23.003 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4032e-05 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541800+0000", 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6638999999999998e-05 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10775 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539075+0000", 2024-09-13T05:02:23.004 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.370977114, 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002929737, 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10775 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539075+0000", 2024-09-13T05:02:23.005 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539075+0000", 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539074+0000", 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.006 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539077+0000", 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539087+0000", 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0696999999999999e-05 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539093+0000", 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.468e-06 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.007 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541834+0000", 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027409029999999998 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541909+0000", 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5204999999999996e-05 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542005+0000", 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5662000000000007e-05 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.008 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10776 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3709547710000001, 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030923719999999999, 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10776 2024-09-13T05:02:23.009 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.010 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539099+0000", 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3149999999999999e-06 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539110+0000", 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0540000000000001e-05 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.011 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539115+0000", 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3449999999999996e-06 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542030+0000", 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002914626 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542103+0000", 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3780000000000002e-05 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.012 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542190+0000", 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6161000000000004e-05 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10825 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:9 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.540367+0000", 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3696853620000002, 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025250200000000002, 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.013 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10825 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540367+0000", 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.014 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540367+0000", 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540366+0000", 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540369+0000", 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.015 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540376+0000", 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1840000000000002e-06 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540381+0000", 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0370000000000002e-06 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542791+0000", 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024095539999999999 2024-09-13T05:02:23.016 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542838+0000", 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7141000000000001e-05 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542892+0000", 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3899e-05 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.017 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10826 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:9 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.540399+0000", 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.369653408, 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027129720000000001, 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10826 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.018 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540399+0000", 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540399+0000", 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540398+0000", 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.019 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540400+0000", 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541020+0000", 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061979099999999996 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541029+0000", 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3839999999999997e-06 2024-09-13T05:02:23.020 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542916+0000", 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001887021 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543004+0000", 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8658000000000004e-05 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543111+0000", 2024-09-13T05:02:23.021 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107142 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10829 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:9 [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3689694609999998, 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002601819, 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.022 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10829 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.023 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541085+0000", 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.537e-06 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541091+0000", 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3289999999999999e-06 2024-09-13T05:02:23.024 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541096+0000", 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9490000000000004e-06 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543497+0000", 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024008509999999999 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.025 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543584+0000", 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7336000000000005e-05 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543684+0000", 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100234 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.026 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.027 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:02:23.029 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:23.030 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:02:23.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:23.032 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-09-13T05:02:22.930633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":188407921,"num_objects":367,"num_object_clones":137,"num_object_copies":734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":367,"num_whiteouts":132,"num_read":25541,"num_read_kb":77997,"num_write":5196,"num_write_kb":29185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3846,"ondisk_log_size":3846,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276944,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":269299,"kb_avail":754697776,"statfs":{"total":773094113280,"available":772810522624,"internally_reserved":0,"allocated":7643136,"data_stored":25710671,"data_compressed":762142,"data_compressed_allocated":3129344,"data_compressed_original":23304028,"omap_allocated":12743,"internal_metadata":275762745},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":41549389,"num_objects":281,"num_object_clones":132,"num_object_copies":562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":281,"num_whiteouts":132,"num_read":17748,"num_read_kb":35302,"num_write":3776,"num_write_kb":12293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.987701"},"pg_stats":[{"pgid":"3.1f","version":"29'96","reported_seq":383,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.009974+0000","last_change":"2024-09-13T05:02:08.892195+0000","last_active":"2024-09-13T05:02:20.009974+0000","last_peered":"2024-09-13T05:02:20.009974+0000","last_clean":"2024-09-13T05:02:20.009974+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:02:20.009974+0000","last_undegraded":"2024-09-13T05:02:20.009974+0000","last_fullsized":"2024-09-13T05:02:20.009974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001008562,"stat_sum":{"num_bytes":1282064,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":582,"num_read_kb":890,"num_write":91,"num_write_kb":313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'176","reported_seq":504,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.935318+0000","last_change":"2024-09-13T05:02:08.935331+0000","last_active":"2024-09-13T05:02:20.935318+0000","last_peered":"2024-09-13T05:02:20.935318+0000","last_clean":"2024-09-13T05:02:20.935318+0000","last_became_active":"2024-09-13T05:02:06.884063+0000","last_became_peered":"2024-09-13T05:02:06.884063+0000","last_unstale":"2024-09-13T05:02:20.935318+0000","last_undegraded":"2024-09-13T05:02:20.935318+0000","last_fullsized":"2024-09-13T05:02:20.935318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:54:39.216716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067803799999999995,"stat_sum":{"num_bytes":1007724,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":909,"num_read_kb":1495,"num_write":292,"num_write_kb":983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"30'87","reported_seq":347,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.067362+0000","last_change":"2024-09-13T05:02:08.887577+0000","last_active":"2024-09-13T05:02:21.067362+0000","last_peered":"2024-09-13T05:02:21.067362+0000","last_clean":"2024-09-13T05:02:21.067362+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:02:21.067362+0000","last_undegraded":"2024-09-13T05:02:21.067362+0000","last_fullsized":"2024-09-13T05:02:21.067362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023315099999999999,"stat_sum":{"num_bytes":839680,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":698,"num_read_kb":600,"num_write":98,"num_write_kb":618,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"30'158","reported_seq":760,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.954683+0000","last_change":"2024-09-13T05:02:08.888247+0000","last_active":"2024-09-13T05:02:20.954683+0000","last_peered":"2024-09-13T05:02:20.954683+0000","last_clean":"2024-09-13T05:02:20.954683+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:02:20.954683+0000","last_undegraded":"2024-09-13T05:02:20.954683+0000","last_fullsized":"2024-09-13T05:02:20.954683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088904499999999998,"stat_sum":{"num_bytes":5264850,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":1087,"num_read_kb":5705,"num_write":274,"num_write_kb":1796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"30'78","reported_seq":1660,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.090024+0000","last_change":"2024-09-13T05:02:08.891592+0000","last_active":"2024-09-13T05:02:21.090024+0000","last_peered":"2024-09-13T05:02:21.090024+0000","last_clean":"2024-09-13T05:02:21.090024+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:02:21.090024+0000","last_undegraded":"2024-09-13T05:02:21.090024+0000","last_fullsized":"2024-09-13T05:02:21.090024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045096999999999997,"stat_sum":{"num_bytes":7308382,"num_objects":12,"num_object_clones":4,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":4,"num_read":1567,"num_read_kb":6298,"num_write":76,"num_write_kb":587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"29'986","reported_seq":1577,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.075279+0000","last_change":"2024-09-13T05:02:08.882149+0000","last_active":"2024-09-13T05:02:21.075279+0000","last_peered":"2024-09-13T05:02:21.075279+0000","last_clean":"2024-09-13T05:02:21.075279+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:02:21.075279+0000","last_undegraded":"2024-09-13T05:02:21.075279+0000","last_fullsized":"2024-09-13T05:02:21.075279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":986,"log_dups_size":0,"ondisk_log_size":986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030432899999999999,"stat_sum":{"num_bytes":10776472,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":5744,"num_read_kb":6459,"num_write":1832,"num_write_kb":3109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"29'42","reported_seq":71,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.931478+0000","last_change":"2024-09-13T05:02:08.882945+0000","last_active":"2024-09-13T05:02:19.931478+0000","last_peered":"2024-09-13T05:02:19.931478+0000","last_clean":"2024-09-13T05:02:19.931478+0000","last_became_active":"2024-09-13T05:02:06.881244+0000","last_became_peered":"2024-09-13T05:02:06.881244+0000","last_unstale":"2024-09-13T05:02:19.931478+0000","last_undegraded":"2024-09-13T05:02:19.931478+0000","last_fullsized":"2024-09-13T05:02:19.931478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:21.706328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078365599999999998,"stat_sum":{"num_bytes":638976,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":10,"num_read_kb":0,"num_write":40,"num_write_kb":181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"29'90","reported_seq":638,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.009201+0000","last_change":"2024-09-13T05:02:08.935674+0000","last_active":"2024-09-13T05:02:20.009201+0000","last_peered":"2024-09-13T05:02:20.009201+0000","last_clean":"2024-09-13T05:02:20.009201+0000","last_became_active":"2024-09-13T05:02:06.880998+0000","last_became_peered":"2024-09-13T05:02:06.880998+0000","last_unstale":"2024-09-13T05:02:20.009201+0000","last_undegraded":"2024-09-13T05:02:20.009201+0000","last_fullsized":"2024-09-13T05:02:20.009201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":90,"log_dups_size":0,"ondisk_log_size":90,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:51:16.614368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089100599999999996,"stat_sum":{"num_bytes":6373376,"num_objects":11,"num_object_clones":5,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":540,"num_read_kb":762,"num_write":86,"num_write_kb":290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"29'67","reported_seq":337,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.011474+0000","last_change":"2024-09-13T05:02:08.892249+0000","last_active":"2024-09-13T05:02:20.011474+0000","last_peered":"2024-09-13T05:02:20.011474+0000","last_clean":"2024-09-13T05:02:20.011474+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:02:20.011474+0000","last_undegraded":"2024-09-13T05:02:20.011474+0000","last_fullsized":"2024-09-13T05:02:20.011474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093220499999999997,"stat_sum":{"num_bytes":10384799,"num_objects":10,"num_object_clones":4,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":256,"num_read_kb":570,"num_write":64,"num_write_kb":461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"29'74","reported_seq":352,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.009094+0000","last_change":"2024-09-13T05:02:08.888457+0000","last_active":"2024-09-13T05:02:20.009094+0000","last_peered":"2024-09-13T05:02:20.009094+0000","last_clean":"2024-09-13T05:02:20.009094+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:02:20.009094+0000","last_undegraded":"2024-09-13T05:02:20.009094+0000","last_fullsized":"2024-09-13T05:02:20.009094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010543060000000001,"stat_sum":{"num_bytes":1815187,"num_objects":9,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":264,"num_read_kb":2145,"num_write":70,"num_write_kb":1250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"29'72","reported_seq":1463,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.930175+0000","last_change":"2024-09-13T05:02:08.934615+0000","last_active":"2024-09-13T05:02:19.930175+0000","last_peered":"2024-09-13T05:02:19.930175+0000","last_clean":"2024-09-13T05:02:19.930175+0000","last_became_active":"2024-09-13T05:02:06.883052+0000","last_became_peered":"2024-09-13T05:02:06.883052+0000","last_unstale":"2024-09-13T05:02:19.930175+0000","last_undegraded":"2024-09-13T05:02:19.930175+0000","last_fullsized":"2024-09-13T05:02:19.930175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:04.845507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042864699999999999,"stat_sum":{"num_bytes":3031040,"num_objects":11,"num_object_clones":5,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1378,"num_read_kb":3865,"num_write":83,"num_write_kb":1716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"29'70","reported_seq":771,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.071281+0000","last_change":"2024-09-13T05:02:08.882243+0000","last_active":"2024-09-13T05:02:21.071281+0000","last_peered":"2024-09-13T05:02:21.071281+0000","last_clean":"2024-09-13T05:02:21.071281+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:02:21.071281+0000","last_undegraded":"2024-09-13T05:02:21.071281+0000","last_fullsized":"2024-09-13T05:02:21.071281+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027033299999999999,"stat_sum":{"num_bytes":4618141,"num_objects":12,"num_object_clones":5,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":696,"num_read_kb":2529,"num_write":65,"num_write_kb":233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.958709+0000","last_change":"2024-09-13T05:01:59.394849+0000","last_active":"2024-09-13T05:02:18.958709+0000","last_peered":"2024-09-13T05:02:18.958709+0000","last_clean":"2024-09-13T05:02:18.958709+0000","last_became_active":"2024-09-13T05:01:57.178293+0000","last_became_peered":"2024-09-13T05:01:57.178293+0000","last_unstale":"2024-09-13T05:02:18.958709+0000","last_undegraded":"2024-09-13T05:02:18.958709+0000","last_fullsized":"2024-09-13T05:02:18.958709+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:03.945702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040222199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"29'69","reported_seq":361,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.008626+0000","last_change":"2024-09-13T05:02:08.888247+0000","last_active":"2024-09-13T05:02:20.008626+0000","last_peered":"2024-09-13T05:02:20.008626+0000","last_clean":"2024-09-13T05:02:20.008626+0000","last_became_active":"2024-09-13T05:02:06.881356+0000","last_became_peered":"2024-09-13T05:02:06.881356+0000","last_unstale":"2024-09-13T05:02:20.008626+0000","last_undegraded":"2024-09-13T05:02:20.008626+0000","last_fullsized":"2024-09-13T05:02:20.008626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:37.032892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085252899999999998,"stat_sum":{"num_bytes":10171103,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":4,"num_read":295,"num_read_kb":1299,"num_write":66,"num_write_kb":932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.958692+0000","last_change":"2024-09-13T05:01:59.394844+0000","last_active":"2024-09-13T05:02:18.958692+0000","last_peered":"2024-09-13T05:02:18.958692+0000","last_clean":"2024-09-13T05:02:18.958692+0000","last_became_active":"2024-09-13T05:01:57.177751+0000","last_became_peered":"2024-09-13T05:01:57.177751+0000","last_unstale":"2024-09-13T05:02:18.958692+0000","last_undegraded":"2024-09-13T05:02:18.958692+0000","last_fullsized":"2024-09-13T05:02:18.958692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:30:48.740698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039273500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"29'121","reported_seq":870,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.007497+0000","last_change":"2024-09-13T05:02:08.883472+0000","last_active":"2024-09-13T05:02:20.007497+0000","last_peered":"2024-09-13T05:02:20.007497+0000","last_clean":"2024-09-13T05:02:20.007497+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:02:20.007497+0000","last_undegraded":"2024-09-13T05:02:20.007497+0000","last_fullsized":"2024-09-13T05:02:20.007497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012097049999999999,"stat_sum":{"num_bytes":8560551,"num_objects":25,"num_object_clones":11,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":11,"num_read":1124,"num_read_kb":2517,"num_write":141,"num_write_kb":800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.919424+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:02:18.919424+0000","last_peered":"2024-09-13T05:02:18.919424+0000","last_clean":"2024-09-13T05:02:18.919424+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:02:18.919424+0000","last_undegraded":"2024-09-13T05:02:18.919424+0000","last_fullsized":"2024-09-13T05:02:18.919424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"29'113","reported_seq":225,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.303068+0000","last_change":"2024-09-13T05:02:08.935465+0000","last_active":"2024-09-13T05:02:19.303068+0000","last_peered":"2024-09-13T05:02:19.303068+0000","last_clean":"2024-09-13T05:02:19.303068+0000","last_became_active":"2024-09-13T05:02:06.884640+0000","last_became_peered":"2024-09-13T05:02:06.884640+0000","last_unstale":"2024-09-13T05:02:19.303068+0000","last_undegraded":"2024-09-13T05:02:19.303068+0000","last_fullsized":"2024-09-13T05:02:19.303068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:51.793989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075041500000000004,"stat_sum":{"num_bytes":3278053,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":281,"num_read_kb":1343,"num_write":129,"num_write_kb":365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.954991+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:02:18.954991+0000","last_peered":"2024-09-13T05:02:18.954991+0000","last_clean":"2024-09-13T05:02:18.954991+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:02:18.954991+0000","last_undegraded":"2024-09-13T05:02:18.954991+0000","last_fullsized":"2024-09-13T05:02:18.954991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"29'52","reported_seq":295,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.929631+0000","last_change":"2024-09-13T05:02:08.891922+0000","last_active":"2024-09-13T05:02:19.929631+0000","last_peered":"2024-09-13T05:02:19.929631+0000","last_clean":"2024-09-13T05:02:19.929631+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:02:19.929631+0000","last_undegraded":"2024-09-13T05:02:19.929631+0000","last_fullsized":"2024-09-13T05:02:19.929631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075461600000000005,"stat_sum":{"num_bytes":929255,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":3,"num_read":230,"num_read_kb":554,"num_write":49,"num_write_kb":314,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"29'137","reported_seq":860,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.931405+0000","last_change":"2024-09-13T05:02:08.891590+0000","last_active":"2024-09-13T05:02:19.931405+0000","last_peered":"2024-09-13T05:02:19.931405+0000","last_clean":"2024-09-13T05:02:19.931405+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:02:19.931405+0000","last_undegraded":"2024-09-13T05:02:19.931405+0000","last_fullsized":"2024-09-13T05:02:19.931405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071068800000000001,"stat_sum":{"num_bytes":8055457,"num_objects":14,"num_object_clones":5,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":5,"num_read":1406,"num_read_kb":2454,"num_write":161,"num_write_kb":644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":174,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.922014+0000","last_change":"2024-09-13T05:01:57.175202+0000","last_active":"2024-09-13T05:02:18.922014+0000","last_peered":"2024-09-13T05:02:18.922014+0000","last_clean":"2024-09-13T05:02:18.922014+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:02:18.922014+0000","last_undegraded":"2024-09-13T05:02:18.922014+0000","last_fullsized":"2024-09-13T05:02:18.922014+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:29.733421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":30,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.925601+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:02:18.925601+0000","last_peered":"2024-09-13T05:02:18.925601+0000","last_clean":"2024-09-13T05:02:18.925601+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:02:18.925601+0000","last_undegraded":"2024-09-13T05:02:18.925601+0000","last_fullsized":"2024-09-13T05:02:18.925601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"30'117","reported_seq":866,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.108214+0000","last_change":"2024-09-13T05:02:08.883195+0000","last_active":"2024-09-13T05:02:21.108214+0000","last_peered":"2024-09-13T05:02:21.108214+0000","last_clean":"2024-09-13T05:02:21.108214+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:02:21.108214+0000","last_undegraded":"2024-09-13T05:02:21.108214+0000","last_fullsized":"2024-09-13T05:02:21.108214+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097658399999999996,"stat_sum":{"num_bytes":6634827,"num_objects":17,"num_object_clones":6,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":1177,"num_read_kb":5676,"num_write":118,"num_write_kb":375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":31,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.922058+0000","last_change":"2024-09-13T05:01:59.195603+0000","last_active":"2024-09-13T05:02:18.922058+0000","last_peered":"2024-09-13T05:02:18.922058+0000","last_clean":"2024-09-13T05:02:18.922058+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:02:18.922058+0000","last_undegraded":"2024-09-13T05:02:18.922058+0000","last_fullsized":"2024-09-13T05:02:18.922058+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:02.276500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"29'56","reported_seq":716,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.722243+0000","last_change":"2024-09-13T05:02:08.882467+0000","last_active":"2024-09-13T05:02:19.722243+0000","last_peered":"2024-09-13T05:02:19.722243+0000","last_clean":"2024-09-13T05:02:19.722243+0000","last_became_active":"2024-09-13T05:02:06.880673+0000","last_became_peered":"2024-09-13T05:02:06.880673+0000","last_unstale":"2024-09-13T05:02:19.722243+0000","last_undegraded":"2024-09-13T05:02:19.722243+0000","last_fullsized":"2024-09-13T05:02:19.722243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":56,"log_dups_size":0,"ondisk_log_size":56,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:45:24.929086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048698900000000001,"stat_sum":{"num_bytes":15545398,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":646,"num_read_kb":11210,"num_write":57,"num_write_kb":617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.932657+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:02:18.932657+0000","last_peered":"2024-09-13T05:02:18.932657+0000","last_clean":"2024-09-13T05:02:18.932657+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:02:18.932657+0000","last_undegraded":"2024-09-13T05:02:18.932657+0000","last_fullsized":"2024-09-13T05:02:18.932657+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"29'118","reported_seq":253,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.031447+0000","last_change":"2024-09-13T05:02:08.934991+0000","last_active":"2024-09-13T05:02:20.031447+0000","last_peered":"2024-09-13T05:02:20.031447+0000","last_clean":"2024-09-13T05:02:20.031447+0000","last_became_active":"2024-09-13T05:02:06.882765+0000","last_became_peered":"2024-09-13T05:02:06.882765+0000","last_unstale":"2024-09-13T05:02:20.031447+0000","last_undegraded":"2024-09-13T05:02:20.031447+0000","last_fullsized":"2024-09-13T05:02:20.031447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:21.053116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045448500000000002,"stat_sum":{"num_bytes":1054940,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":437,"num_read_kb":734,"num_write":163,"num_write_kb":374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:18.922067+0000","last_change":"2024-09-13T05:01:59.195613+0000","last_active":"2024-09-13T05:02:18.922067+0000","last_peered":"2024-09-13T05:02:18.922067+0000","last_clean":"2024-09-13T05:02:18.922067+0000","last_became_active":"2024-09-13T05:01:57.172867+0000","last_became_peered":"2024-09-13T05:01:57.172867+0000","last_unstale":"2024-09-13T05:02:18.922067+0000","last_undegraded":"2024-09-13T05:02:18.922067+0000","last_fullsized":"2024-09-13T05:02:18.922067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:03:56.676979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033540799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"29'99","reported_seq":984,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.014061+0000","last_change":"2024-09-13T05:02:08.935586+0000","last_active":"2024-09-13T05:02:20.014061+0000","last_peered":"2024-09-13T05:02:20.014061+0000","last_clean":"2024-09-13T05:02:20.014061+0000","last_became_active":"2024-09-13T05:02:06.881316+0000","last_became_peered":"2024-09-13T05:02:06.881316+0000","last_unstale":"2024-09-13T05:02:20.014061+0000","last_undegraded":"2024-09-13T05:02:20.014061+0000","last_fullsized":"2024-09-13T05:02:20.014061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:07.292949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084766500000000005,"stat_sum":{"num_bytes":3975350,"num_objects":20,"num_object_clones":8,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":8,"num_read":889,"num_read_kb":2463,"num_write":144,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"29'73","reported_seq":297,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.007178+0000","last_change":"2024-09-13T05:02:08.882379+0000","last_active":"2024-09-13T05:02:20.007178+0000","last_peered":"2024-09-13T05:02:20.007178+0000","last_clean":"2024-09-13T05:02:20.007178+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:02:20.007178+0000","last_undegraded":"2024-09-13T05:02:20.007178+0000","last_fullsized":"2024-09-13T05:02:20.007178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039834799999999997,"stat_sum":{"num_bytes":5137492,"num_objects":9,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":4,"num_read":216,"num_read_kb":684,"num_write":90,"num_write_kb":889,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"30'102","reported_seq":326,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.971236+0000","last_change":"2024-09-13T05:02:08.934869+0000","last_active":"2024-09-13T05:02:20.971236+0000","last_peered":"2024-09-13T05:02:20.971236+0000","last_clean":"2024-09-13T05:02:20.971236+0000","last_became_active":"2024-09-13T05:02:06.887529+0000","last_became_peered":"2024-09-13T05:02:06.887529+0000","last_unstale":"2024-09-13T05:02:20.971236+0000","last_undegraded":"2024-09-13T05:02:20.971236+0000","last_fullsized":"2024-09-13T05:02:20.971236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:43.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050065899999999998,"stat_sum":{"num_bytes":1181859,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":412,"num_read_kb":860,"num_write":144,"num_write_kb":1706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"30'47","reported_seq":216,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.058061+0000","last_change":"2024-09-13T05:02:08.934984+0000","last_active":"2024-09-13T05:02:21.058061+0000","last_peered":"2024-09-13T05:02:21.058061+0000","last_clean":"2024-09-13T05:02:21.058061+0000","last_became_active":"2024-09-13T05:02:06.881094+0000","last_became_peered":"2024-09-13T05:02:06.881094+0000","last_unstale":"2024-09-13T05:02:21.058061+0000","last_undegraded":"2024-09-13T05:02:21.058061+0000","last_fullsized":"2024-09-13T05:02:21.058061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:02:56.840152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046295800000000002,"stat_sum":{"num_bytes":2265286,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":157,"num_read_kb":1065,"num_write":44,"num_write_kb":616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"29'83","reported_seq":405,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.009718+0000","last_change":"2024-09-13T05:02:08.887634+0000","last_active":"2024-09-13T05:02:20.009718+0000","last_peered":"2024-09-13T05:02:20.009718+0000","last_clean":"2024-09-13T05:02:20.009718+0000","last_became_active":"2024-09-13T05:02:06.879695+0000","last_became_peered":"2024-09-13T05:02:06.879695+0000","last_unstale":"2024-09-13T05:02:20.009718+0000","last_undegraded":"2024-09-13T05:02:20.009718+0000","last_fullsized":"2024-09-13T05:02:20.009718+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:23.915465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028084900000000001,"stat_sum":{"num_bytes":917544,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":720,"num_read_kb":906,"num_write":76,"num_write_kb":512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"29'49","reported_seq":620,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.008155+0000","last_change":"2024-09-13T05:02:08.935026+0000","last_active":"2024-09-13T05:02:20.008155+0000","last_peered":"2024-09-13T05:02:20.008155+0000","last_clean":"2024-09-13T05:02:20.008155+0000","last_became_active":"2024-09-13T05:02:06.885421+0000","last_became_peered":"2024-09-13T05:02:06.885421+0000","last_unstale":"2024-09-13T05:02:20.008155+0000","last_undegraded":"2024-09-13T05:02:20.008155+0000","last_fullsized":"2024-09-13T05:02:20.008155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:20:39.514148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071617299999999996,"stat_sum":{"num_bytes":5902336,"num_objects":9,"num_object_clones":4,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":564,"num_read_kb":6823,"num_write":50,"num_write_kb":1154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"30'35","reported_seq":68,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.086104+0000","last_change":"2024-09-13T05:02:08.934818+0000","last_active":"2024-09-13T05:02:21.086104+0000","last_peered":"2024-09-13T05:02:21.086104+0000","last_clean":"2024-09-13T05:02:21.086104+0000","last_became_active":"2024-09-13T05:02:06.881683+0000","last_became_peered":"2024-09-13T05:02:06.881683+0000","last_unstale":"2024-09-13T05:02:21.086104+0000","last_undegraded":"2024-09-13T05:02:21.086104+0000","last_fullsized":"2024-09-13T05:02:21.086104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:25:51.679049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057826299999999998,"stat_sum":{"num_bytes":4089397,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":20,"num_read_kb":73,"num_write":35,"num_write_kb":243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"29'49","reported_seq":336,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:19.929822+0000","last_change":"2024-09-13T05:02:08.934811+0000","last_active":"2024-09-13T05:02:19.929822+0000","last_peered":"2024-09-13T05:02:19.929822+0000","last_clean":"2024-09-13T05:02:19.929822+0000","last_became_active":"2024-09-13T05:02:06.884741+0000","last_became_peered":"2024-09-13T05:02:06.884741+0000","last_unstale":"2024-09-13T05:02:19.929822+0000","last_undegraded":"2024-09-13T05:02:19.929822+0000","last_fullsized":"2024-09-13T05:02:19.929822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:29:29.184441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053391699999999996,"stat_sum":{"num_bytes":11963476,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":274,"num_read_kb":569,"num_write":48,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"30'89","reported_seq":347,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.924805+0000","last_change":"2024-09-13T05:02:08.934690+0000","last_active":"2024-09-13T05:02:20.924805+0000","last_peered":"2024-09-13T05:02:20.924805+0000","last_clean":"2024-09-13T05:02:20.924805+0000","last_became_active":"2024-09-13T05:02:06.884642+0000","last_became_peered":"2024-09-13T05:02:06.884642+0000","last_unstale":"2024-09-13T05:02:20.924805+0000","last_undegraded":"2024-09-13T05:02:20.924805+0000","last_fullsized":"2024-09-13T05:02:20.924805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:18.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000322436,"stat_sum":{"num_bytes":8306688,"num_objects":14,"num_object_clones":6,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":6,"num_read":249,"num_read_kb":740,"num_write":135,"num_write_kb":1354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"30'110","reported_seq":587,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.099852+0000","last_change":"2024-09-13T05:02:08.934351+0000","last_active":"2024-09-13T05:02:21.099852+0000","last_peered":"2024-09-13T05:02:21.099852+0000","last_clean":"2024-09-13T05:02:21.099852+0000","last_became_active":"2024-09-13T05:02:06.884670+0000","last_became_peered":"2024-09-13T05:02:06.884670+0000","last_unstale":"2024-09-13T05:02:21.099852+0000","last_undegraded":"2024-09-13T05:02:21.099852+0000","last_fullsized":"2024-09-13T05:02:21.099852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:15.372629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028879200000000001,"stat_sum":{"num_bytes":20566507,"num_objects":21,"num_object_clones":8,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":8,"num_read":535,"num_read_kb":1415,"num_write":100,"num_write_kb":615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"30'150","reported_seq":993,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-13T05:02:21.077708+0000","last_change":"2024-09-13T05:02:08.891841+0000","last_active":"2024-09-13T05:02:21.077708+0000","last_peered":"2024-09-13T05:02:21.077708+0000","last_clean":"2024-09-13T05:02:21.077708+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:02:21.077708+0000","last_undegraded":"2024-09-13T05:02:21.077708+0000","last_fullsized":"2024-09-13T05:02:21.077708+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":150,"log_dups_size":0,"ondisk_log_size":150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068869099999999996,"stat_sum":{"num_bytes":7963296,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1681,"num_read_kb":4039,"num_write":145,"num_write_kb":633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"29'61","reported_seq":455,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-09-13T05:02:20.010137+0000","last_change":"2024-09-13T05:02:08.934570+0000","last_active":"2024-09-13T05:02:20.010137+0000","last_peered":"2024-09-13T05:02:20.010137+0000","last_clean":"2024-09-13T05:02:20.010137+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:02:20.010137+0000","last_undegraded":"2024-09-13T05:02:20.010137+0000","last_fullsized":"2024-09-13T05:02:20.010137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042496100000000001,"stat_sum":{"num_bytes":7978028,"num_objects":11,"num_object_clones":4,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":381,"num_read_kb":1155,"num_write":69,"num_write_kb":1333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":187817534,"num_objects":363,"num_object_clones":137,"num_object_copies":726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":363,"num_whiteouts":132,"num_read":25425,"num_read_kb":77897,"num_write":5035,"num_write_kb":26767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":5111808,"data_stored":23662572,"data_compressed":768088,"data_compressed_allocated":3096576,"data_compressed_original":22263644,"omap_allocated":0,"internal_metadata":0},"log_size":3726,"ondisk_log_size":3726,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1069056,"data_stored":8197455,"data_compressed":193448,"data_compressed_allocated":434176,"data_compressed_original":7843840,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":593920,"data_stored":669315,"data_compressed":1645,"data_compressed_allocated":192512,"data_compressed_original":479232,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":806912,"data_stored":4759470,"data_compressed":102356,"data_compressed_allocated":282624,"data_compressed_original":4493312,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40596,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331244,"statfs":{"total":96636764160,"available":96595193856,"internally_reserved":0,"allocated":1314816,"data_stored":2498706,"data_compressed":132213,"data_compressed_allocated":577536,"data_compressed_original":2073725,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36408,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335432,"statfs":{"total":96636764160,"available":96599482368,"internally_reserved":0,"allocated":1179648,"data_stored":3877480,"data_compressed":135211,"data_compressed_allocated":557056,"data_compressed_original":3553073,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":585728,"data_stored":638649,"data_compressed":16633,"data_compressed_allocated":163840,"data_compressed_original":431229,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36308,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94335532,"statfs":{"total":96636764160,"available":96599584768,"internally_reserved":0,"allocated":1114112,"data_stored":2323556,"data_compressed":75164,"data_compressed_allocated":544768,"data_compressed_original":2012977,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.214}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":978944,"data_stored":2746040,"data_compressed":105472,"data_compressed_allocated":376832,"data_compressed_original":2416640,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":692224,"data_stored":2658726,"data_compressed":105542,"data_compressed_allocated":385024,"data_compressed_original":2441216,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":749568,"data_stored":1610112,"data_compressed":70313,"data_compressed_allocated":507904,"data_compressed_original":1439537,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":515573,"data_compressed":16633,"data_compressed_allocated":163840,"data_compressed_original":431229,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":880640,"data_stored":3766994,"data_compressed":142930,"data_compressed_allocated":565248,"data_compressed_original":3565361,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1024000,"data_stored":2415772,"data_compressed":132318,"data_compressed_allocated":589824,"data_compressed_original":2114685,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":4633129,"data_compressed":102356,"data_compressed_allocated":282624,"data_compressed_original":4493312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":274432,"data_stored":541302,"data_compressed":1645,"data_compressed_allocated":192512,"data_compressed_original":479232,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":720896,"data_stored":7520964,"data_compressed":196351,"data_compressed_allocated":409600,"data_compressed_original":7299072,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:02:23.034 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [2, 5, 6, 7] 2024-09-13T05:02:23.035 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.15', '2', '5', '6', '7'] 2024-09-13T05:02:23.035 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.15 2 5 6 7 2024-09-13T05:02:23.035 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:23.027+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:23.129 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:23.128+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:23.138 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:23.138 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.146 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:23.191 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone8 from 0x1ef1b8a (overlap 0x1161cab) to 0xa6de00 2024-09-13T05:02:23.230 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:23.230+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:23.251 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-13T05:02:23.254 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:23.254 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:23.254 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:23.255 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:23.255 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:23.331 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:23.331+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8383 3.10 3:0d0d9ed8:::rbd_data.117c6c62fa22.000000000000017a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:23.373 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.247621+0000", 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.122524941, 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022406259999999999, 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8383 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247621+0000", 2024-09-13T05:02:23.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247621+0000", 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247620+0000", 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247625+0000", 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7929999999999997e-06 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247636+0000", 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0541e-05 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247641+0000", 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2750000000000003e-06 2024-09-13T05:02:23.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247685+0000", 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3639e-05 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247765+0000", 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0663999999999999e-05 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247862+0000", 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6521000000000006e-05 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.248559+0000", 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00069733800000000004 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.249794+0000", 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001235002 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.249826+0000", 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2081e-05 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.249862+0000", 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5670000000000002e-05 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8615 3.10 3:0cc2e45c:::rbd_data.117c6c62fa22.000000000000003a:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.404628+0000", 2024-09-13T05:02:23.379 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.9655180259999998, 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044752689999999996, 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8615 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404628+0000", 2024-09-13T05:02:23.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404628+0000", 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404626+0000", 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999976 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404637+0000", 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0669999999999999e-05 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404655+0000", 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8067000000000002e-05 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404664+0000", 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.329e-06 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404712+0000", 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7691000000000001e-05 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404818+0000", 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010636099999999999 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.405694+0000", 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087614500000000005 2024-09-13T05:02:23.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.408785+0000", 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030904639999999998 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409049+0000", 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00026435 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409081+0000", 2024-09-13T05:02:23.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2152e-05 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409103+0000", 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2289999999999998e-05 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8710 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [set-alloc-hint object_size 524288 write_size 524288,write 217088~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.458384+0000", 2024-09-13T05:02:23.385 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.9117621509999996, 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026806640000000001, 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8710 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458384+0000", 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458384+0000", 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458382+0000", 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458395+0000", 2024-09-13T05:02:23.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2634e-05 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458411+0000", 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6489000000000001e-05 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458420+0000", 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7260000000000008e-06 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458474+0000", 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3769e-05 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458557+0000", 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2984000000000004e-05 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458699+0000", 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014173500000000001 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.459521+0000", 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00082235700000000001 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.460987+0000", 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001466237 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.461029+0000", 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2188000000000003e-05 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.461065+0000", 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5321000000000003e-05 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8882 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.560284+0000", 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.8098623319999998, 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023403019999999998, 2024-09-13T05:02:23.391 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8882 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560284+0000", 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560284+0000", 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560283+0000", 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560292+0000", 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1260000000000004e-06 2024-09-13T05:02:23.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560303+0000", 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.041e-05 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560308+0000", 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3750000000000002e-06 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560346+0000", 2024-09-13T05:02:23.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8031999999999998e-05 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560425+0000", 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8768000000000002e-05 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560543+0000", 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001182 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.561304+0000", 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00076059800000000002 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.562548+0000", 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001243971 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.562592+0000", 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4764e-05 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.562624+0000", 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1850000000000002e-05 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8970 3.a 3:547253cd:::rbd_data.117c6c62fa22.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.808539+0000", 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.5616071099999997, 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028653709999999998, 2024-09-13T05:02:23.397 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8970 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808539+0000", 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808539+0000", 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808537+0000", 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808552+0000", 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4345000000000001e-05 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808566+0000", 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4552000000000001e-05 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808574+0000", 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7069999999999992e-06 2024-09-13T05:02:23.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808620+0000", 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6023999999999998e-05 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808746+0000", 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012644800000000001 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808882+0000", 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000135747 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.809502+0000", 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062012700000000005 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.811319+0000", 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018165919999999999 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.811367+0000", 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8491e-05 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.811404+0000", 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7092999999999999e-05 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9252 3.10 3:0817f0cd:::rbd_data.117c6c62fa22.00000000000001b9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.994069+0000", 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.3760771380000003, 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021901999999999998, 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9252 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.404 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994069+0000", 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994069+0000", 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994068+0000", 2024-09-13T05:02:23.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994073+0000", 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4079999999999997e-06 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994084+0000", 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1054e-05 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994090+0000", 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2059999999999998e-06 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994136+0000", 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6904999999999999e-05 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994214+0000", 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7708999999999997e-05 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994317+0000", 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103284 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.994906+0000", 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058843000000000003 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.996211+0000", 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013052439999999999 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.996238+0000", 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6375000000000001e-05 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.996259+0000", 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1755000000000001e-05 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9276 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.012014+0000", 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.3581323029999997, 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029183199999999999, 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9276 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012014+0000", 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012014+0000", 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012013+0000", 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012018+0000", 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9189999999999998e-06 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012028+0000", 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0199e-05 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012034+0000", 2024-09-13T05:02:23.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012070+0000", 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6396999999999997e-05 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012145+0000", 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4802000000000007e-05 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012564+0000", 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041880599999999997 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014635+0000", 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020714489999999999 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014884+0000", 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024929299999999999 2024-09-13T05:02:23.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014912+0000", 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8209e-05 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014932+0000", 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9779000000000002e-05 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9401 3.10 3:0902317e:::rbd_data.117c6c62fa22.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.093414+0000", 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.2767320550000001, 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024931250000000001, 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9401 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093414+0000", 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093414+0000", 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093413+0000", 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093422+0000", 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7339999999999997e-06 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093435+0000", 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3278000000000001e-05 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093443+0000", 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4599999999999997e-06 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093489+0000", 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6377e-05 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093568+0000", 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9433999999999999e-05 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093663+0000", 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4806e-05 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094246+0000", 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058259100000000003 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095828+0000", 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001582186 2024-09-13T05:02:23.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095873+0000", 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5460000000000002e-05 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095907+0000", 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3899999999999997e-05 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9410 3.10 3:0d7acefb:::rbd_data.117c6c62fa22.000000000000010a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.094592+0000", 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.2755547109999998, 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023194359999999998, 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9410 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094592+0000", 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094592+0000", 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094591+0000", 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094596+0000", 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.288e-06 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094608+0000", 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2378000000000001e-05 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094619+0000", 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0706999999999999e-05 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094739+0000", 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120098 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094823+0000", 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3424000000000006e-05 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094899+0000", 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6810000000000005e-05 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095401+0000", 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050169600000000004 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.096843+0000", 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001441583 2024-09-13T05:02:23.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.096885+0000", 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2052999999999999e-05 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.096911+0000", 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6256999999999999e-05 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.427 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9515 3.a 3:5630f440:::rbd_data.117c6c62fa22.000000000000006b:head [set-alloc-hint object_size 524288 write_size 524288,write 186907~1509] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.2095898260000002, 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003723638, 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9515 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160561+0000", 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160572+0000", 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0845999999999999e-05 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160578+0000", 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7050000000000002e-06 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160609+0000", 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0664000000000003e-05 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160683+0000", 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4041000000000005e-05 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160756+0000", 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2745999999999998e-05 2024-09-13T05:02:23.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.161214+0000", 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045812700000000001 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.164194+0000", 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00298008 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.164239+0000", 2024-09-13T05:02:23.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5402999999999999e-05 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.164280+0000", 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0717999999999999e-05 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9786 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [set-alloc-hint object_size 524288 write_size 524288,write 172032~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.297975+0000", 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.0721716020000001, 2024-09-13T05:02:23.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0037219729999999999, 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9786 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297975+0000", 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297975+0000", 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297973+0000", 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297991+0000", 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8079000000000001e-05 2024-09-13T05:02:23.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298011+0000", 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0489e-05 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298021+0000", 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6509999999999996e-06 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298067+0000", 2024-09-13T05:02:23.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5596999999999997e-05 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298138+0000", 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0932000000000005e-05 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298277+0000", 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013981200000000001 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.299096+0000", 2024-09-13T05:02:23.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00081815299999999996 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.301606+0000", 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025103830000000001 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.301658+0000", 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1974000000000001e-05 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.301697+0000", 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8661999999999997e-05 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9878 3.a 3:542814f5:::rbd_data.117c6c62fa22.000000000000009d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.355033+0000", 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.0151131580000001, 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070892680000000001, 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.439 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9878 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355033+0000", 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355033+0000", 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355031+0000", 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355042+0000", 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0468e-05 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355058+0000", 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6905000000000002e-05 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355066+0000", 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355121+0000", 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4586000000000002e-05 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355197+0000", 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6236999999999995e-05 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355603+0000", 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000405876 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.358219+0000", 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026158959999999999 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362058+0000", 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00383928 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362097+0000", 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8685e-05 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362122+0000", 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5435999999999998e-05 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9882 3.10 3:0d194f86:::rbd_data.117c6c62fa22.00000000000000a2:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.362188+0000", 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.0079585059999996, 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029458700000000002, 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9882 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.446 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362188+0000", 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362188+0000", 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362186+0000", 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362193+0000", 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1729999999999998e-06 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362206+0000", 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2116000000000001e-05 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362211+0000", 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9860000000000002e-06 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362253+0000", 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.269e-05 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362393+0000", 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000139951 2024-09-13T05:02:23.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362889+0000", 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049619300000000002 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364752+0000", 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001862608 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.365043+0000", 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029077500000000001 2024-09-13T05:02:23.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.365093+0000", 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0680000000000003e-05 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.365134+0000", 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0117999999999998e-05 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4476.0:10084 3.1e e29/19 3:7bb5a129:::rbd_data.117c6c62fa22.0000000000000001:head v 29'51, mlcod=28'48)", 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.977835+0000", 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3923108690000001, 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002452503, 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977835+0000", 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977835+0000", 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977834+0000", 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977845+0000", 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0953e-05 2024-09-13T05:02:23.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977862+0000", 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6376e-05 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977869+0000", 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977947+0000", 2024-09-13T05:02:23.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7533000000000005e-05 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977971+0000", 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4165999999999999e-05 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980254+0000", 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002283205 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980288+0000", 2024-09-13T05:02:23.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3469000000000001e-05 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10093 3.10 3:0efb4843:::rbd_data.117c6c62fa22.000000000000000a:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.980322+0000", 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3898241569999996, 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029339129999999998, 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10093 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980322+0000", 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980322+0000", 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980321+0000", 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980326+0000", 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8980000000000001e-06 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980339+0000", 2024-09-13T05:02:23.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2782000000000001e-05 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980346+0000", 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0690000000000004e-06 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980396+0000", 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9403000000000001e-05 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980478+0000", 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1834000000000003e-05 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980627+0000", 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014940299999999999 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.982097+0000", 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001469896 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.983207+0000", 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011104470000000001 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.983233+0000", 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5383000000000001e-05 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.983256+0000", 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3308000000000001e-05 2024-09-13T05:02:23.461 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10122 3.a 3:547253cd:::rbd_data.117c6c62fa22.000000000000002a:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.989485+0000", 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3806609400000003, 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022556690000000001, 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10122 2024-09-13T05:02:23.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989485+0000", 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989485+0000", 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989483+0000", 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989492+0000", 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6889999999999993e-06 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989512+0000", 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0024999999999999e-05 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989521+0000", 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2879999999999998e-06 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989557+0000", 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5500999999999998e-05 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989621+0000", 2024-09-13T05:02:23.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4333999999999998e-05 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989740+0000", 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011927800000000001 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990479+0000", 2024-09-13T05:02:23.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00073908000000000003 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991647+0000", 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001167916 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991693+0000", 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5414000000000003e-05 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991741+0000", 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8143999999999998e-05 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10138 3.10 3:0cc2e45c:::rbd_data.117c6c62fa22.000000000000003a:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.993508+0000", 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3766381589999996, 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022845640000000002, 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.468 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10138 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993508+0000", 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993508+0000", 2024-09-13T05:02:23.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993507+0000", 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993513+0000", 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5470000000000002e-06 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993528+0000", 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4773999999999999e-05 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993536+0000", 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7090000000000002e-06 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993574+0000", 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8152000000000003e-05 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993633+0000", 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9630000000000003e-05 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993778+0000", 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000144771 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994625+0000", 2024-09-13T05:02:23.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084671200000000001 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.995706+0000", 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001080777 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.995753+0000", 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-05 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.995793+0000", 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9158e-05 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10189 3.a 3:55f43daa:::rbd_data.117c6c62fa22.00000000000000b4:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.006631+0000", 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3635147849999996, 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022169070000000002, 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10189 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006631+0000", 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006631+0000", 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006630+0000", 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006637+0000", 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9110000000000004e-06 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006655+0000", 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7969e-05 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006663+0000", 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7800000000000001e-06 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006709+0000", 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6270999999999998e-05 2024-09-13T05:02:23.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006768+0000", 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9134e-05 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006912+0000", 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014435000000000001 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007791+0000", 2024-09-13T05:02:23.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087889399999999996 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008772+0000", 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00098034800000000003 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008816+0000", 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3986999999999997e-05 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008848+0000", 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2663000000000002e-05 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10192 3.10 3:0c2bfc09:::rbd_data.117c6c62fa22.0000000000000106:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.007406+0000", 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.3627405670000003, 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027390090000000001, 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10192 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007406+0000", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007406+0000", 2024-09-13T05:02:23.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007405+0000", 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007414+0000", 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9509999999999995e-06 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007428+0000", 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4450000000000001e-05 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007445+0000", 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6739e-05 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007498+0000", 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2859000000000003e-05 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007585+0000", 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7253e-05 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007755+0000", 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000169772 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008565+0000", 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080988900000000005 2024-09-13T05:02:23.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010064+0000", 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014997389999999999 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010111+0000", 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6657999999999999e-05 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010145+0000", 2024-09-13T05:02:23.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3631999999999999e-05 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:20060 3.10 3:085258df:::rbd_object_map.117c7aa26b99:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:22.968786+0000", 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.40136042500000002, 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002612376, 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.486 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 20060 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968786+0000", 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968786+0000", 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968784+0000", 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968793+0000", 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8340000000000005e-06 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968808+0000", 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5037e-05 2024-09-13T05:02:23.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968815+0000", 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6019999999999996e-06 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968881+0000", 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6126999999999999e-05 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968887+0000", 2024-09-13T05:02:23.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1070000000000001e-06 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970175+0000", 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001287814 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970242+0000", 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7735000000000001e-05 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970347+0000", 2024-09-13T05:02:23.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104975 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970870+0000", 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052217500000000005 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.971351+0000", 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048108899999999997 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.971378+0000", 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7727999999999999e-05 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.971398+0000", 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.982e-05 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.492 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.493 INFO:teuthology.orchestra.run.smithi053.stdout:1079 trunc from 0x1ef1b8a to 0x6be2469 2024-09-13T05:02:23.494 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:23.494 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:23.493+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:23.496 INFO:teuthology.orchestra.run.smithi053.stdout:1081 read 0xe563d7 thru 0xe584e2 (0x210c bytes) 2024-09-13T05:02:23.496 INFO:teuthology.orchestra.run.smithi053.stdout:1082 read 0x1177c33 thru 0x117c0f4 (0x44c2 bytes) 2024-09-13T05:02:23.498 INFO:teuthology.orchestra.run.smithi053.stdout:1083 trunc from 0x6be2469 to 0x7be4b42 2024-09-13T05:02:23.503 INFO:teuthology.orchestra.run.smithi053.stdout:1084 writesame 0x1e9d43d thru 0x1ea62e4 data_size 0x16(0x8ea8 bytes) 2024-09-13T05:02:23.507 INFO:teuthology.orchestra.run.smithi053.stdout:1085 read 0xd96c7c thru 0xd9e99e (0x7d23 bytes) 2024-09-13T05:02:23.508 INFO:teuthology.orchestra.run.smithi053.stdout:1086 read 0x648040 thru 0x657c65 (0xfc26 bytes) 2024-09-13T05:02:23.508 INFO:teuthology.orchestra.run.smithi053.stdout:1087 writesame 0x37d1344 thru 0x37d70f2 data_size 0x1(0x5daf bytes) 2024-09-13T05:02:23.535 INFO:teuthology.orchestra.run.smithi053.stdout:1088 writesame 0xe9be2ab thru 0xe9c4aed data_size 0x29(0x6843 bytes) 2024-09-13T05:02:23.535 INFO:teuthology.orchestra.run.smithi053.stdout:1089 punch from 0xdbe357d to 0xdbe76c4, (0x4147 bytes) 2024-09-13T05:02:23.535 INFO:teuthology.orchestra.run.smithi053.stdout:1090 write 0x434256 thru 0x4414f9 (0xd2a4 bytes) 2024-09-13T05:02:23.542 INFO:teuthology.orchestra.run.smithi053.stdout:1091 punch from 0x547387a to 0x547a05f, (0x67e5 bytes) 2024-09-13T05:02:23.542 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.542 INFO:teuthology.orchestra.run.smithi053.stdout:1095 read 0x74b8e6e thru 0x74c7696 (0xe829 bytes) 2024-09-13T05:02:23.542 INFO:teuthology.orchestra.run.smithi053.stdout:1097 writesame 0xd86cb9e thru 0xd86e6b1 data_size 0x2(0x1b14 bytes) 2024-09-13T05:02:23.546 INFO:teuthology.orchestra.run.smithi053.stdout:1101 write 0xe4f5ddc thru 0xe503d72 (0xdf97 bytes) 2024-09-13T05:02:23.549 INFO:teuthology.orchestra.run.smithi053.stdout:1102 read 0x35ea0c3 thru 0x35ec0d8 (0x2016 bytes) 2024-09-13T05:02:23.550 INFO:teuthology.orchestra.run.smithi053.stdout:1103 read 0x7da089a thru 0x7da09df (0x146 bytes) 2024-09-13T05:02:23.550 INFO:teuthology.orchestra.run.smithi053.stdout:1104 writesame 0x6687227 thru 0x668cae7 data_size 0x1(0x58c1 bytes) 2024-09-13T05:02:23.562 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.563 INFO:teuthology.orchestra.run.smithi053.stdout:1107 trunc from 0xe9c4aee to 0xdd0b312 2024-09-13T05:02:23.577 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.577 INFO:teuthology.orchestra.run.smithi053.stdout:1109 read 0xc236fd4 thru 0xc243cc8 (0xccf5 bytes) 2024-09-13T05:02:23.577 INFO:teuthology.orchestra.run.smithi053.stdout:1110 write 0x84249d1 thru 0x8427dae (0x33de bytes) 2024-09-13T05:02:23.577 INFO:teuthology.orchestra.run.smithi053.stdout:1112 write 0xaf45ac6 thru 0xaf4d985 (0x7ec0 bytes) 2024-09-13T05:02:23.579 INFO:teuthology.orchestra.run.smithi053.stdout:1113 writesame 0x647520c thru 0x647f601 data_size 0x1f(0xa3f6 bytes) 2024-09-13T05:02:23.583 INFO:teuthology.orchestra.run.smithi053.stdout:1114 read 0x8bbc2b thru 0x8cb085 (0xf45b bytes) 2024-09-13T05:02:23.583 INFO:teuthology.orchestra.run.smithi053.stdout:1115 read 0xb05d9dd thru 0xb06d28e (0xf8b2 bytes) 2024-09-13T05:02:23.591 INFO:teuthology.orchestra.run.smithi053.stdout:1116 trunc from 0xdd0b312 to 0x6ddb111 2024-09-13T05:02:23.594 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:23.594+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:23.601 INFO:teuthology.orchestra.run.smithi053.stdout:1117 punch from 0x1ca5528 to 0x1cabd83, (0x685b bytes) 2024-09-13T05:02:23.601 INFO:teuthology.orchestra.run.smithi053.stdout:1118 read 0x507be8d thru 0x508234f (0x64c3 bytes) 2024-09-13T05:02:23.601 INFO:teuthology.orchestra.run.smithi053.stdout:1119 writesame 0x48994ca thru 0x48a48b0 data_size 0x3d(0xb3e7 bytes) 2024-09-13T05:02:23.604 INFO:teuthology.orchestra.run.smithi053.stdout:1120 punch from 0x243f14 to 0x25342e, (0xf51a bytes) 2024-09-13T05:02:23.604 INFO:teuthology.orchestra.run.smithi053.stdout:1121 read 0x1616943 thru 0x16236cf (0xcd8d bytes) 2024-09-13T05:02:23.615 INFO:teuthology.orchestra.run.smithi053.stdout:1122 write 0xb3f9be3 thru 0xb3fe605 (0x4a23 bytes) 2024-09-13T05:02:23.618 INFO:teuthology.orchestra.run.smithi053.stdout:1124 punch from 0x7a0345 to 0x7a8bbf, (0x887a bytes) 2024-09-13T05:02:23.630 INFO:teuthology.orchestra.run.smithi053.stdout:1125 trunc from 0xb3fe606 to 0x657b51 2024-09-13T05:02:23.641 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.641 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.645 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:23.645 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:23.655 INFO:teuthology.orchestra.run.smithi053.stdout:1129 write 0x8410173 thru 0x84145ff (0x448d bytes) 2024-09-13T05:02:23.657 INFO:teuthology.orchestra.run.smithi053.stdout:1130 writesame 0x13cc6ab thru 0x13cde77 data_size 0x3(0x17cd bytes) 2024-09-13T05:02:23.661 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.661 INFO:teuthology.orchestra.run.smithi053.stdout:1134 write 0x5c9350e thru 0x5c9cba7 (0x969a bytes) 2024-09-13T05:02:23.664 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.664 INFO:teuthology.orchestra.run.smithi053.stdout:1136 read 0x45ac73e thru 0x45b1130 (0x49f3 bytes) 2024-09-13T05:02:23.664 INFO:teuthology.orchestra.run.smithi053.stdout:1138 read 0x27593e1 thru 0x27624fc (0x911c bytes) 2024-09-13T05:02:23.674 INFO:teuthology.orchestra.run.smithi053.stdout:1140 write 0xc3366eb thru 0xc340369 (0x9c7f bytes) 2024-09-13T05:02:23.677 INFO:teuthology.orchestra.run.smithi053.stdout:1141 write 0x1d5e192 thru 0x1d6ba14 (0xd883 bytes) 2024-09-13T05:02:23.680 INFO:teuthology.orchestra.run.smithi053.stdout:1142 read 0x7cf949a thru 0x7d009d3 (0x753a bytes) 2024-09-13T05:02:23.680 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:23.689 INFO:teuthology.orchestra.run.smithi053.stdout:1145 write 0xee273c9 thru 0xee34db8 (0xd9f0 bytes) 2024-09-13T05:02:23.692 INFO:teuthology.orchestra.run.smithi053.stdout:1146 write 0xcfd7295 thru 0xcfd77f9 (0x565 bytes) 2024-09-13T05:02:23.695 INFO:teuthology.orchestra.run.smithi053.stdout:1149 write 0xdf13b06 thru 0xdf21d70 (0xe26b bytes) 2024-09-13T05:02:23.695 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:23.695+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:23.698 INFO:teuthology.orchestra.run.smithi053.stdout:1150 read 0xb42e92c thru 0xb438a51 (0xa126 bytes) 2024-09-13T05:02:23.698 INFO:teuthology.orchestra.run.smithi053.stdout:1151 write 0x54e1f73 thru 0x54e5767 (0x37f5 bytes) 2024-09-13T05:02:23.701 INFO:teuthology.orchestra.run.smithi053.stdout:1152 punch from 0x2d32a10 to 0x2d38675, (0x5c65 bytes) 2024-09-13T05:02:23.701 INFO:teuthology.orchestra.run.smithi053.stdout:1153 read 0x8ae67f6 thru 0x8aecc56 (0x6461 bytes) 2024-09-13T05:02:23.701 INFO:teuthology.orchestra.run.smithi053.stdout:1154 clone 10 order 21 su 8192 sc 12 2024-09-13T05:02:23.758 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:23.758 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:23.758 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:23.758 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:23.758 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.766 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:23.810 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:23.810+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:36 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:23.871 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.66655621, 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042082860000000003, 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 36 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202322+0000", 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202325+0000", 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2619999999999999e-06 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202329+0000", 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7500000000000001e-06 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202332+0000", 2024-09-13T05:02:23.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2600000000000001e-06 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205205+0000", 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028723030000000001 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205253+0000", 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8458000000000003e-05 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205337+0000", 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4499999999999994e-05 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206379+0000", 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010417250000000001 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206460+0000", 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1100999999999995e-05 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206499+0000", 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8398000000000001e-05 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206531+0000", 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1917e-05 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.877 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:37 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.666542593999999, 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044315279999999997, 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 37 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202336+0000", 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202338+0000", 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1210000000000001e-06 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202341+0000", 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2389999999999998e-06 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.202344+0000", 2024-09-13T05:02:23.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6340000000000001e-06 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205834+0000", 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034905420000000001 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.205928+0000", 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3741999999999998e-05 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206061+0000", 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013342599999999999 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206545+0000", 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048334400000000003 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206699+0000", 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015441399999999999 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206730+0000", 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1143999999999999e-05 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.206767+0000", 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7128000000000001e-05 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.883 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380202+0000", 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488676884, 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041373750000000004, 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 110 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380202+0000", 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380202+0000", 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380201+0000", 2024-09-13T05:02:23.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380207+0000", 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4869999999999999e-06 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380216+0000", 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2530000000000004e-06 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380221+0000", 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2870000000000004e-06 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382907+0000", 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002685607 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.382962+0000", 2024-09-13T05:02:23.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5504000000000003e-05 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383035+0000", 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2608999999999995e-05 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384278+0000", 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012430659999999999 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384294+0000", 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5854000000000001e-05 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384319+0000", 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5174999999999999e-05 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384339+0000", 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0313e-05 2024-09-13T05:02:23.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488651135000001, 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0045436620000000004, 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.890 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 111 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380227+0000", 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380231+0000", 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5870000000000001e-06 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380237+0000", 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0850000000000002e-06 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380249+0000", 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2037e-05 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383346+0000", 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003096901 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383390+0000", 2024-09-13T05:02:23.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3937999999999998e-05 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383468+0000", 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8220000000000007e-05 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384346+0000", 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087789499999999996 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384726+0000", 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000380388 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384751+0000", 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5398999999999999e-05 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384771+0000", 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9661999999999999e-05 2024-09-13T05:02:23.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380256+0000", 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488622764999999, 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0054419129999999996, 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.896 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 112 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380256+0000", 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380256+0000", 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380255+0000", 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380305+0000", 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9851000000000001e-05 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380314+0000", 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4479999999999994e-06 2024-09-13T05:02:23.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380319+0000", 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4489999999999998e-06 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383689+0000", 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033696519999999999 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383737+0000", 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7870000000000001e-05 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.383820+0000", 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3602000000000004e-05 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385610+0000", 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001789674 2024-09-13T05:02:23.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385649+0000", 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9461e-05 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385678+0000", 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8773e-05 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385698+0000", 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9664999999999999e-05 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488546491000001, 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0054681809999999999, 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 113 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380332+0000", 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380336+0000", 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2230000000000001e-06 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380341+0000", 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380352+0000", 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1785e-05 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384363+0000", 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0040111089999999997 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384409+0000", 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5799e-05 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384479+0000", 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9468000000000006e-05 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385739+0000", 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012602539999999999 2024-09-13T05:02:23.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385745+0000", 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4579999999999998e-06 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385781+0000", 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5728000000000001e-05 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385800+0000", 2024-09-13T05:02:23.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9072999999999998e-05 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380359+0000", 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488519772, 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058786300000000001, 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.908 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 114 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380359+0000", 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380359+0000", 2024-09-13T05:02:23.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380358+0000", 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380363+0000", 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3069999999999996e-06 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380375+0000", 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2128e-05 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380380+0000", 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8300000000000003e-06 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384798+0000", 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044188609999999996 2024-09-13T05:02:23.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384852+0000", 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3226000000000002e-05 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.384919+0000", 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7354000000000006e-05 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385807+0000", 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00088828300000000002 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386188+0000", 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00038091299999999998 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386217+0000", 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8513e-05 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386237+0000", 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0695000000000001e-05 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488489149999999, 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.006229315, 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.914 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 115 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380389+0000", 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380431+0000", 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1675000000000003e-05 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380440+0000", 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8910000000000007e-06 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380447+0000", 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7320000000000001e-06 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385156+0000", 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0047083309999999996 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385200+0000", 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385282+0000", 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2164999999999997e-05 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386271+0000", 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00098910800000000004 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386566+0000", 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029463699999999999 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386600+0000", 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3603999999999999e-05 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386619+0000", 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9143999999999998e-05 2024-09-13T05:02:23.919 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380455+0000", 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488423839999999, 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063589129999999999, 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.920 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 116 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380455+0000", 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380455+0000", 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380454+0000", 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380458+0000", 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2560000000000004e-06 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380465+0000", 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1940000000000003e-06 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380469+0000", 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9550000000000002e-06 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385828+0000", 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0053589689999999999 2024-09-13T05:02:23.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385876+0000", 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8522000000000001e-05 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.385952+0000", 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5629000000000004e-05 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386627+0000", 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00067552300000000003 2024-09-13T05:02:23.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386771+0000", 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014388500000000001 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386794+0000", 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3008999999999999e-05 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386814+0000", 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9497999999999999e-05 2024-09-13T05:02:23.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.488406159, 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066318890000000002, 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.926 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 117 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380472+0000", 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380476+0000", 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1130000000000003e-06 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380482+0000", 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9290000000000003e-06 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.380486+0000", 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999999e-06 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386288+0000", 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0058020939999999998 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386334+0000", 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6069000000000002e-05 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386411+0000", 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6779000000000005e-05 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.386820+0000", 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00040942399999999998 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.387063+0000", 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024340000000000001 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.387087+0000", 2024-09-13T05:02:23.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4037e-05 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.387104+0000", 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6918999999999998e-05 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:135 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779131+0000", 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089747892999998, 2024-09-13T05:02:23.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042690239999999997, 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 135 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779131+0000", 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779131+0000", 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779130+0000", 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779142+0000", 2024-09-13T05:02:23.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2517999999999999e-05 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779149+0000", 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0319999999999998e-06 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779155+0000", 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3260000000000003e-06 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.781636+0000", 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024811550000000001 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.781713+0000", 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7763999999999996e-05 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.781811+0000", 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7412999999999997e-05 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782690+0000", 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087884300000000003 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783317+0000", 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062709199999999999 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783363+0000", 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6310999999999998e-05 2024-09-13T05:02:23.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783400+0000", 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6569999999999997e-05 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089717255, 2024-09-13T05:02:23.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0043404610000000003, 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 136 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779161+0000", 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779165+0000", 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7620000000000001e-06 2024-09-13T05:02:23.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779171+0000", 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5150000000000003e-06 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779176+0000", 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8289999999999998e-06 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782109+0000", 2024-09-13T05:02:23.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029328179999999998 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782161+0000", 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782230+0000", 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9307999999999994e-05 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783412+0000", 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011818309999999999 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783450+0000", 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8118000000000003e-05 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783482+0000", 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2329000000000002e-05 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783502+0000", 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9522e-05 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089697594, 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0051891749999999999, 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 137 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779181+0000", 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779224+0000", 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3047000000000003e-05 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779229+0000", 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3260000000000003e-06 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779232+0000", 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.219e-06 2024-09-13T05:02:23.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782715+0000", 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003482855 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782764+0000", 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8566999999999998e-05 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.948 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.782848+0000", 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4756e-05 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784277+0000", 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001428971 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784286+0000", 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7099999999999996e-06 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784333+0000", 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6882000000000003e-05 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784370+0000", 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7175000000000002e-05 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.950 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089642411, 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063116589999999998, 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 138 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779236+0000", 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779238+0000", 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1940000000000001e-06 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779241+0000", 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.157e-06 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779244+0000", 2024-09-13T05:02:23.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783534+0000", 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0042906410000000004 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783597+0000", 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3026000000000003e-05 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783669+0000", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1044999999999995e-05 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785428+0000", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001759622 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785471+0000", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3305e-05 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785513+0000", 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1820000000000003e-05 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785548+0000", 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4535000000000001e-05 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.956 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089632249000001, 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0063959350000000002, 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 139 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.957 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779246+0000", 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779248+0000", 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1890000000000001e-06 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779253+0000", 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6489999999999999e-06 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779256+0000", 2024-09-13T05:02:23.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.88e-06 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783899+0000", 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0046426979999999998 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.783950+0000", 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1665000000000001e-05 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784019+0000", 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9187e-05 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785588+0000", 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0015683610000000001 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785593+0000", 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.916e-06 2024-09-13T05:02:23.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785623+0000", 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0128999999999999e-05 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785642+0000", 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9471e-05 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779261+0000", 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089617899, 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0064895719999999999, 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 140 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.963 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779261+0000", 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779261+0000", 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779260+0000", 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779310+0000", 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9830999999999998e-05 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779317+0000", 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0849999999999999e-06 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779321+0000", 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2820000000000001e-06 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784408+0000", 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0050876330000000003 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784483+0000", 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4759999999999996e-05 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.784606+0000", 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000122617 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785648+0000", 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001042304 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785682+0000", 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.358e-05 2024-09-13T05:02:23.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785719+0000", 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7645999999999998e-05 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785750+0000", 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1053000000000002e-05 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4213.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:01:57.779325+0000", 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 26.089553606999999, 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0066314759999999999, 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4213", 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.110:0/4031988898", 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 141 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779325+0000", 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779325+0000", 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779324+0000", 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.970 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779328+0000", 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4960000000000001e-06 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779334+0000", 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6500000000000001e-06 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.779340+0000", 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0510000000000002e-06 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785069+0000", 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0057296600000000001 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785113+0000", 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3838999999999998e-05 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785186+0000", 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2831999999999997e-05 2024-09-13T05:02:23.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785760+0000", 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00057425800000000002 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785886+0000", 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012604099999999999 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785922+0000", 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5710999999999997e-05 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:01:57.785956+0000", 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4397000000000003e-05 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.974 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:1954 3.11 3:891a5151:::rbd_data.117cdd702390.0000000000000083:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.469736+0000", 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.3991424460000008, 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.012593695, 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1954 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469736+0000", 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469736+0000", 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469735+0000", 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469741+0000", 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7049999999999996e-06 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469751+0000", 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0689e-05 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469756+0000", 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0499999999999999e-06 2024-09-13T05:02:23.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469802+0000", 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5178e-05 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.469940+0000", 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013883799999999999 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.471960+0000", 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020193110000000002 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.482263+0000", 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010303100000000001 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.482270+0000", 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9650000000000002e-06 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.979 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.482301+0000", 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0859999999999999e-05 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.482330+0000", 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9172000000000001e-05 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.980 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:1991 3.13 3:cd702726:::rbd_data.117cdd702390.000000000000008a:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.517796+0000", 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.3510820849999998, 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0041741440000000003, 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1991 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.981 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517796+0000", 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517796+0000", 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517796+0000", 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.982 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517801+0000", 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9150000000000004e-06 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517813+0000", 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.257e-05 2024-09-13T05:02:23.984 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517822+0000", 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5250000000000005e-06 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.517871+0000", 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9252999999999998e-05 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.985 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.518004+0000", 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000133115 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.518698+0000", 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000694182 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.521856+0000", 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031583420000000002 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.986 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.521907+0000", 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0874000000000001e-05 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.521946+0000", 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8899999999999997e-05 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.521971+0000", 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4349e-05 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.987 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:20654 3.f 3:f26fe086:::rbd_data.117c7aa26b99.000000000000000d:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:23.511071+0000", 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.357807296, 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010854572, 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 20654 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:23.988 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511071+0000", 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511071+0000", 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511070+0000", 2024-09-13T05:02:23.989 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511076+0000", 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3109999999999996e-06 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511086+0000", 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.058e-05 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:23.990 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511092+0000", 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3970000000000002e-06 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511124+0000", 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2153999999999997e-05 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.511206+0000", 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2528000000000007e-05 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.991 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.513187+0000", 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001980838 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.521833+0000", 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0086458239999999999 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.521843+0000", 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0053e-05 2024-09-13T05:02:23.992 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.521889+0000", 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5824e-05 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.521926+0000", 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6989000000000001e-05 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:23.993 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:23.994 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:23.994 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:23.995 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:02:23.996 INFO:teuthology.orchestra.run.smithi053.stderr:set 3.15 pg_upmap_items mapping to [2->5,6->7] 2024-09-13T05:02:23.997 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:23.998 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:23.994+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:24.002 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone9 intact 2024-09-13T05:02:24.096 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:24.096+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:24.120 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:24.120 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:24.120 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:24.134 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:24.141 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:24.141 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:24.160 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:24.160 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:24.176 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-13T05:02:24.179 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:24.179 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:24.194 INFO:teuthology.orchestra.run.smithi053.stdout:1155 write 0xbc1174d thru 0xbc1e5c6 (0xce7a bytes) 2024-09-13T05:02:24.196 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:24.196+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:24.197 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:24.198 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:24.208 INFO:teuthology.orchestra.run.smithi053.stdout:1156 writesame 0x4061e5f thru 0x406aa9c data_size 0x1d(0x8c3e bytes) 2024-09-13T05:02:24.216 INFO:teuthology.orchestra.run.smithi053.stdout:1157 punch from 0x7eadbdd to 0x7eb9377, (0xb79a bytes) 2024-09-13T05:02:24.216 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:24.217 INFO:teuthology.orchestra.run.smithi053.stdout:1159 trunc from 0xee34db9 to 0xd9f6c3e 2024-09-13T05:02:24.220 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:24.220 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:24.246 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:24.247 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:24.268 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:24.268 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:24.269 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:24.270 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:24.280 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:24.292 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:24.292 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:24.303 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:24.303+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:24.321 INFO:teuthology.orchestra.run.smithi053.stdout:1162 read 0x1d6cac1 thru 0x1d79ac5 (0xd005 bytes) 2024-09-13T05:02:24.330 INFO:teuthology.orchestra.run.smithi053.stdout:1163 trunc from 0xd9f6c3e to 0x648db0e 2024-09-13T05:02:24.367 INFO:teuthology.orchestra.run.smithi053.stdout:1164 write 0xa784c54 thru 0xa78e4b3 (0x9860 bytes) 2024-09-13T05:02:24.373 INFO:teuthology.orchestra.run.smithi053.stdout:1166 punch from 0x1ad8af1 to 0x1ae0510, (0x7a1f bytes) 2024-09-13T05:02:24.373 INFO:teuthology.orchestra.run.smithi053.stdout:1170 read 0x6c0aa8d thru 0x6c0f3cb (0x493f bytes) 2024-09-13T05:02:24.380 INFO:teuthology.orchestra.run.smithi053.stdout:1171 writesame 0xc839f00 thru 0xc83b865 data_size 0x1(0x1966 bytes) 2024-09-13T05:02:24.383 INFO:teuthology.orchestra.run.smithi053.stdout:1172 trunc from 0xc83b866 to 0xebe1f22 2024-09-13T05:02:24.388 INFO:teuthology.orchestra.run.smithi053.stdout:1173 read 0x558c183 thru 0x559095f (0x47dd bytes) 2024-09-13T05:02:24.388 INFO:teuthology.orchestra.run.smithi053.stdout:1174 read 0x2786bc0 thru 0x278fe9e (0x92df bytes) 2024-09-13T05:02:24.388 INFO:teuthology.orchestra.run.smithi053.stdout:1176 write 0xb401aa3 thru 0xb40d91a (0xbe78 bytes) 2024-09-13T05:02:24.396 INFO:teuthology.orchestra.run.smithi053.stdout:1177 punch from 0x3864f8 to 0x393020, (0xcb28 bytes) 2024-09-13T05:02:24.396 INFO:teuthology.orchestra.run.smithi053.stdout:1178 write 0xcd131ce thru 0xcd1f922 (0xc755 bytes) 2024-09-13T05:02:24.398 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:24.398+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:24.404 INFO:teuthology.orchestra.run.smithi053.stdout:1179 read 0x663bdfa thru 0x663d61d (0x1824 bytes) 2024-09-13T05:02:24.405 INFO:teuthology.orchestra.run.smithi053.stdout:1180 punch from 0x6b74c4e to 0x6b8071a, (0xbacc bytes) 2024-09-13T05:02:24.405 INFO:teuthology.orchestra.run.smithi053.stdout:1181 punch from 0xdb9cb14 to 0xdb9f9c0, (0x2eac bytes) 2024-09-13T05:02:24.406 INFO:teuthology.orchestra.run.smithi053.stdout:1182 writesame 0x5771002 thru 0x577f49c data_size 0x1(0xe49b bytes) 2024-09-13T05:02:24.423 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:24.424 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:24.436 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:24.487 INFO:teuthology.orchestra.run.smithi053.stdout:1183 read 0x59b5da7 thru 0x59bf2fe (0x9558 bytes) 2024-09-13T05:02:24.487 INFO:teuthology.orchestra.run.smithi053.stdout:1184 read 0x117f0a8 thru 0x1183681 (0x45da bytes) 2024-09-13T05:02:24.488 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:24.488 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:24.488 INFO:teuthology.orchestra.run.smithi053.stdout:1188 write 0xbb378b3 thru 0xbb3a6fa (0x2e48 bytes) 2024-09-13T05:02:24.490 INFO:teuthology.orchestra.run.smithi053.stdout:1189 punch from 0x7cda1b2 to 0x7ce9a40, (0xf88e bytes) 2024-09-13T05:02:24.491 INFO:teuthology.orchestra.run.smithi053.stdout:1190 write 0x88d8178 thru 0x88e53ac (0xd235 bytes) 2024-09-13T05:02:24.499 INFO:teuthology.orchestra.run.smithi053.stdout:1191 clone 11 order 24 su 262144 sc 7 2024-09-13T05:02:24.499 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:24.498+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:24.586 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:24.586 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:24.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:24.637 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:24.636+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:24.700 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:24.700+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:24.742 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:24.742 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:24.751 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:24.836 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:24.835+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:24.898 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:24.898 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:24.902 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:24.901+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:24.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:25.002 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:25.002+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:25.010 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone10 intact 2024-09-13T05:02:25.061 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.061 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.071 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:25.115 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:25.115+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:25.184 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.185 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.187 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-13T05:02:25.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:25.237 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:25.236+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:25.304 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:25.304+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:25.351 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.351 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:25.405 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:25.405+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:25.444 INFO:teuthology.orchestra.run.smithi053.stdout:1194 read 0x66410e5 thru 0x6649652 (0x856e bytes) 2024-09-13T05:02:25.445 INFO:teuthology.orchestra.run.smithi053.stdout:1195 write 0x4282b6 thru 0x435750 (0xd49b bytes) 2024-09-13T05:02:25.478 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.478 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:25.506 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:25.506+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:25.528 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.528 INFO:teuthology.orchestra.run.smithi053.stdout:1197 read 0xe0056f0 thru 0xe013e48 (0xe759 bytes) 2024-09-13T05:02:25.528 INFO:teuthology.orchestra.run.smithi053.stdout:1198 read 0x30457b2 thru 0x304706a (0x18b9 bytes) 2024-09-13T05:02:25.529 INFO:teuthology.orchestra.run.smithi053.stdout:1200 read 0x33723fe thru 0x337a0c8 (0x7ccb bytes) 2024-09-13T05:02:25.529 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.529 INFO:teuthology.orchestra.run.smithi053.stdout:1203 write 0xc630b5d thru 0xc63211e (0x15c2 bytes) 2024-09-13T05:02:25.566 INFO:teuthology.orchestra.run.smithi053.stdout:1204 writesame 0x6e7c851 thru 0x6e7f78e data_size 0x1(0x2f3e bytes) 2024-09-13T05:02:25.607 INFO:teuthology.orchestra.run.smithi053.stdout:1205 writesame 0x15807b3 thru 0x1589ba1 data_size 0x1(0x93ef bytes) 2024-09-13T05:02:25.607 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:25.606+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:25.612 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.612 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.621 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:25.636 INFO:teuthology.orchestra.run.smithi053.stdout:1206 trunc from 0xebe1f22 to 0x34c80ee 2024-09-13T05:02:25.708 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:25.708+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:25.781 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.782 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.792 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:25.812 INFO:teuthology.orchestra.run.smithi053.stdout:1207 punch from 0x271b844 to 0x2723a8f, (0x824b bytes) 2024-09-13T05:02:25.813 INFO:teuthology.orchestra.run.smithi053.stdout:1208 punch from 0x1b6432b to 0x1b679a6, (0x367b bytes) 2024-09-13T05:02:25.813 INFO:teuthology.orchestra.run.smithi053.stdout:1209 punch from 0x477bc9 to 0x482fc1, (0xb3f8 bytes) 2024-09-13T05:02:25.813 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:25.812+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:25.823 INFO:teuthology.orchestra.run.smithi053.stdout:1210 writesame 0x718cf6a thru 0x718e387 data_size 0x19(0x141e bytes) 2024-09-13T05:02:25.823 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.837 INFO:teuthology.orchestra.run.smithi053.stdout:1212 write 0xdf23f7a thru 0xdf307a7 (0xc82e bytes) 2024-09-13T05:02:25.840 INFO:teuthology.orchestra.run.smithi053.stdout:1215 punch from 0x5dd9699 to 0x5de8a3b, (0xf3a2 bytes) 2024-09-13T05:02:25.840 INFO:teuthology.orchestra.run.smithi053.stdout:1216 write 0x181e47a thru 0x181f8dc (0x1463 bytes) 2024-09-13T05:02:25.841 INFO:teuthology.orchestra.run.smithi053.stdout:1217 read 0x9845676 thru 0x984e71d (0x90a8 bytes) 2024-09-13T05:02:25.842 INFO:teuthology.orchestra.run.smithi053.stdout:1218 trunc from 0xdf307a8 to 0xdd591ba 2024-09-13T05:02:25.849 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.849 INFO:teuthology.orchestra.run.smithi053.stdout:1222 read 0x9d4c39e thru 0x9d4d355 (0xfb8 bytes) 2024-09-13T05:02:25.849 INFO:teuthology.orchestra.run.smithi053.stdout:1223 write 0xabdb811 thru 0xabdf776 (0x3f66 bytes) 2024-09-13T05:02:25.851 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.859 INFO:teuthology.orchestra.run.smithi053.stdout:1226 write 0xe8c9e78 thru 0xe8d02a4 (0x642d bytes) 2024-09-13T05:02:25.861 INFO:teuthology.orchestra.run.smithi053.stdout:1227 read 0x75f2419 thru 0x75f8742 (0x632a bytes) 2024-09-13T05:02:25.862 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.862 INFO:teuthology.orchestra.run.smithi053.stdout:1230 punch from 0x7c34d2c to 0x7c3efb7, (0xa28b bytes) 2024-09-13T05:02:25.869 INFO:teuthology.orchestra.run.smithi053.stdout:1231 trunc from 0xe8d02a5 to 0x79844c5 2024-09-13T05:02:25.880 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.891 INFO:teuthology.orchestra.run.smithi053.stdout:1234 writesame 0xc2619c7 thru 0xc26ec99 data_size 0x1(0xd2d3 bytes) 2024-09-13T05:02:25.891 INFO:teuthology.orchestra.run.smithi053.stdout:1235 punch from 0xbf21cd to 0xbfd506, (0xb339 bytes) 2024-09-13T05:02:25.891 INFO:teuthology.orchestra.run.smithi053.stdout:1236 write 0x214e20e thru 0x2154b2f (0x6922 bytes) 2024-09-13T05:02:25.892 INFO:teuthology.orchestra.run.smithi053.stdout:1237 read 0x3980b7f thru 0x398608c (0x550e bytes) 2024-09-13T05:02:25.900 INFO:teuthology.orchestra.run.smithi053.stdout:1238 trunc from 0xc26ec9a to 0x57aba5e 2024-09-13T05:02:25.903 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:25.903 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:25.910 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:25.910+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:25.913 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:25.921 INFO:teuthology.orchestra.run.smithi053.stdout:1240 trunc from 0x57aba5e to 0xddf1789 2024-09-13T05:02:25.927 INFO:teuthology.orchestra.run.smithi053.stdout:1241 read 0x564a383 thru 0x564d1b5 (0x2e33 bytes) 2024-09-13T05:02:25.927 INFO:teuthology.orchestra.run.smithi053.stdout:1242 read 0x2b82d9e thru 0x2b92b4a (0xfdad bytes) 2024-09-13T05:02:25.928 INFO:teuthology.orchestra.run.smithi053.stdout:1244 write 0x4b47d63 thru 0x4b4959c (0x183a bytes) 2024-09-13T05:02:25.929 INFO:teuthology.orchestra.run.smithi053.stdout:1245 write 0xb23b84f thru 0xb23f5e6 (0x3d98 bytes) 2024-09-13T05:02:25.932 INFO:teuthology.orchestra.run.smithi053.stdout:1246 write 0x14152c5 thru 0x1419a0c (0x4748 bytes) 2024-09-13T05:02:25.936 INFO:teuthology.orchestra.run.smithi053.stdout:1247 trunc from 0xddf1789 to 0xc1ba998 2024-09-13T05:02:25.943 INFO:teuthology.orchestra.run.smithi053.stdout:1248 read 0x1b7cd75 thru 0x1b898e4 (0xcb70 bytes) 2024-09-13T05:02:25.943 INFO:teuthology.orchestra.run.smithi053.stdout:1249 writesame 0x30283ab thru 0x3037267 data_size 0x1(0xeebd bytes) 2024-09-13T05:02:25.970 INFO:teuthology.orchestra.run.smithi053.stdout:1250 punch from 0x4579d to 0x4b02c, (0x588f bytes) 2024-09-13T05:02:25.970 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:25.970 INFO:teuthology.orchestra.run.smithi053.stdout:1253 punch from 0x1463084 to 0x146c6d4, (0x9650 bytes) 2024-09-13T05:02:25.973 INFO:teuthology.orchestra.run.smithi053.stdout:1255 trunc from 0xc1ba998 to 0x9a8c0db 2024-09-13T05:02:25.980 INFO:teuthology.orchestra.run.smithi053.stdout:1257 read 0x5706a1a thru 0x570d092 (0x6679 bytes) 2024-09-13T05:02:25.984 INFO:teuthology.orchestra.run.smithi053.stdout:1258 trunc from 0x9a8c0db to 0xd1ceefd 2024-09-13T05:02:25.990 INFO:teuthology.orchestra.run.smithi053.stdout:1259 write 0x3c66bef thru 0x3c74843 (0xdc55 bytes) 2024-09-13T05:02:25.991 INFO:teuthology.orchestra.run.smithi053.stdout:1261 write 0x812ee51 thru 0x812fec6 (0x1076 bytes) 2024-09-13T05:02:25.993 INFO:teuthology.orchestra.run.smithi053.stdout:1262 read 0x96d1d8e thru 0x96dad5b (0x8fce bytes) 2024-09-13T05:02:25.993 INFO:teuthology.orchestra.run.smithi053.stdout:1263 punch from 0x4718b6e to 0x472859d, (0xfa2f bytes) 2024-09-13T05:02:25.993 INFO:teuthology.orchestra.run.smithi053.stdout:1266 write 0x2e6b35f thru 0x2e77b35 (0xc7d7 bytes) 2024-09-13T05:02:25.994 INFO:teuthology.orchestra.run.smithi053.stdout:1267 read 0x91603b7 thru 0x916e9ab (0xe5f5 bytes) 2024-09-13T05:02:25.994 INFO:teuthology.orchestra.run.smithi053.stdout:1270 read 0xa62ac3f thru 0xa62d406 (0x27c8 bytes) 2024-09-13T05:02:25.994 INFO:teuthology.orchestra.run.smithi053.stdout:1274 write 0x24bb0f2 thru 0x24c786e (0xc77d bytes) 2024-09-13T05:02:25.996 INFO:teuthology.orchestra.run.smithi053.stdout:1275 writesame 0x54f5b4a thru 0x55009b9 data_size 0x8(0xae70 bytes) 2024-09-13T05:02:25.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:02:25.999 INFO:teuthology.orchestra.run.smithi053.stdout:1276 read 0x3422449 thru 0x3425971 (0x3529 bytes) 2024-09-13T05:02:25.999 INFO:teuthology.orchestra.run.smithi053.stdout:1277 writesame 0x747e531 thru 0x7480206 data_size 0x1(0x1cd6 bytes) 2024-09-13T05:02:26.005 INFO:teuthology.orchestra.run.smithi053.stdout:1279 punch from 0x9682330 to 0x96915ec, (0xf2bc bytes) 2024-09-13T05:02:26.005 INFO:teuthology.orchestra.run.smithi053.stdout:1280 write 0x23f4300 thru 0x23faef6 (0x6bf7 bytes) 2024-09-13T05:02:26.007 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.007 INFO:teuthology.orchestra.run.smithi053.stdout:1283 read 0x821dbfc thru 0x82225bb (0x49c0 bytes) 2024-09-13T05:02:26.007 INFO:teuthology.orchestra.run.smithi053.stdout:1284 read 0x46173d7 thru 0x4625cf7 (0xe921 bytes) 2024-09-13T05:02:26.007 INFO:teuthology.orchestra.run.smithi053.stdout:1285 writesame 0x8739bd1 thru 0x87421f6 data_size 0xe(0x8626 bytes) 2024-09-13T05:02:26.011 INFO:teuthology.orchestra.run.smithi053.stdout:1286 punch from 0x8ca1026 to 0x8ca78c9, (0x68a3 bytes) 2024-09-13T05:02:26.011 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.012 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:26.011+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:26.015 INFO:teuthology.orchestra.run.smithi053.stdout:1288 trunc from 0xd1ceefd to 0x9c1eecf 2024-09-13T05:02:26.024 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:26.024 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:26.033 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:26.034 INFO:teuthology.orchestra.run.smithi053.stdout:1291 write 0xe7704c7 thru 0xe77b228 (0xad62 bytes) 2024-09-13T05:02:26.037 INFO:teuthology.orchestra.run.smithi053.stdout:1292 punch from 0x36e07fe to 0x36e11c4, (0x9c6 bytes) 2024-09-13T05:02:26.037 INFO:teuthology.orchestra.run.smithi053.stdout:1293 write 0x3cb58c3 thru 0x3cbc2a6 (0x69e4 bytes) 2024-09-13T05:02:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:1294 punch from 0xda9476b to 0xdaa004a, (0xb8df bytes) 2024-09-13T05:02:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.038 INFO:teuthology.orchestra.run.smithi053.stdout:1298 writesame 0x5f2b79e thru 0x5f31d68 data_size 0xb(0x65cb bytes) 2024-09-13T05:02:26.040 INFO:teuthology.orchestra.run.smithi053.stdout:1299 read 0xbdee4b8 thru 0xbdfb50f (0xd058 bytes) 2024-09-13T05:02:26.040 INFO:teuthology.orchestra.run.smithi053.stdout:1301 write 0xe2a6933 thru 0xe2b2c73 (0xc341 bytes) 2024-09-13T05:02:26.044 INFO:teuthology.orchestra.run.smithi053.stdout:1302 punch from 0xbbac544 to 0xbbafb11, (0x35cd bytes) 2024-09-13T05:02:26.044 INFO:teuthology.orchestra.run.smithi053.stdout:1304 read 0x6abc8a4 thru 0x6ac205f (0x57bc bytes) 2024-09-13T05:02:26.044 INFO:teuthology.orchestra.run.smithi053.stdout:1305 write 0x80155b3 thru 0x802267e (0xd0cc bytes) 2024-09-13T05:02:26.046 INFO:teuthology.orchestra.run.smithi053.stdout:1306 read 0x1fed436 thru 0x1ffb60c (0xe1d7 bytes) 2024-09-13T05:02:26.046 INFO:teuthology.orchestra.run.smithi053.stdout:1307 punch from 0xcb0c2fd to 0xcb0c599, (0x29c bytes) 2024-09-13T05:02:26.046 INFO:teuthology.orchestra.run.smithi053.stdout:1308 write 0x89da8c thru 0x8ac340 (0xe8b5 bytes) 2024-09-13T05:02:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:1310 read 0x9a75a4c thru 0x9a7cda3 (0x7358 bytes) 2024-09-13T05:02:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:1311 read 0xb78cc8a thru 0xb792cf0 (0x6067 bytes) 2024-09-13T05:02:26.048 INFO:teuthology.orchestra.run.smithi053.stdout:1312 read 0x41f50e3 thru 0x41f8ac0 (0x39de bytes) 2024-09-13T05:02:26.049 INFO:teuthology.orchestra.run.smithi053.stdout:1313 read 0xc37df06 thru 0xc389bf7 (0xbcf2 bytes) 2024-09-13T05:02:26.049 INFO:teuthology.orchestra.run.smithi053.stdout:1314 punch from 0x20279b9 to 0x20302db, (0x8922 bytes) 2024-09-13T05:02:26.049 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.049 INFO:teuthology.orchestra.run.smithi053.stdout:1316 punch from 0x61f499e to 0x61fb42f, (0x6a91 bytes) 2024-09-13T05:02:26.049 INFO:teuthology.orchestra.run.smithi053.stdout:1317 write 0x3c7e7d6 thru 0x3c81a92 (0x32bd bytes) 2024-09-13T05:02:26.050 INFO:teuthology.orchestra.run.smithi053.stdout:1318 read 0x7cc270a thru 0x7ccf34f (0xcc46 bytes) 2024-09-13T05:02:26.051 INFO:teuthology.orchestra.run.smithi053.stdout:1320 punch from 0x6032b27 to 0x603ba77, (0x8f50 bytes) 2024-09-13T05:02:26.051 INFO:teuthology.orchestra.run.smithi053.stdout:1321 write 0x2fbac61 thru 0x2fc8b60 (0xdf00 bytes) 2024-09-13T05:02:26.053 INFO:teuthology.orchestra.run.smithi053.stdout:1322 read 0x518d915 thru 0x5198c54 (0xb340 bytes) 2024-09-13T05:02:26.053 INFO:teuthology.orchestra.run.smithi053.stdout:1323 read 0x83359ed thru 0x8336178 (0x78c bytes) 2024-09-13T05:02:26.053 INFO:teuthology.orchestra.run.smithi053.stdout:1324 writesame 0x7ad517a thru 0x7ad866a data_size 0x1(0x34f1 bytes) 2024-09-13T05:02:26.060 INFO:teuthology.orchestra.run.smithi053.stdout:1327 writesame 0x595d729 thru 0x596d1fb data_size 0x1(0xfad3 bytes) 2024-09-13T05:02:26.089 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.101 INFO:teuthology.orchestra.run.smithi053.stdout:1329 trunc from 0xe77b229 to 0x35f4f75 2024-09-13T05:02:26.112 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:26.112+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:26.132 INFO:teuthology.orchestra.run.smithi053.stdout:1330 writesame 0xe63d9d6 thru 0xe643b8b data_size 0x21(0x61b6 bytes) 2024-09-13T05:02:26.132 INFO:teuthology.orchestra.run.smithi053.stdout:1333 write 0xd1f7f78 thru 0xd2010d6 (0x915f bytes) 2024-09-13T05:02:26.137 INFO:teuthology.orchestra.run.smithi053.stdout:1334 read 0x58a1b95 thru 0x58aae4c (0x92b8 bytes) 2024-09-13T05:02:26.137 INFO:teuthology.orchestra.run.smithi053.stdout:1336 read 0x2db8fbb thru 0x2dc41a1 (0xb1e7 bytes) 2024-09-13T05:02:26.137 INFO:teuthology.orchestra.run.smithi053.stdout:1338 write 0x71baab1 thru 0x71c23c0 (0x7910 bytes) 2024-09-13T05:02:26.138 INFO:teuthology.orchestra.run.smithi053.stdout:1339 writesame 0x923cd81 thru 0x9247c92 data_size 0x1(0xaf12 bytes) 2024-09-13T05:02:26.142 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:26.142 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:26.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:26.168 INFO:teuthology.orchestra.run.smithi053.stdout:1340 trunc from 0xe643b8c to 0x2a7e5e5 2024-09-13T05:02:26.180 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.180 INFO:teuthology.orchestra.run.smithi053.stdout:1342 read 0x10ffa48 thru 0x110ca89 (0xd042 bytes) 2024-09-13T05:02:26.180 INFO:teuthology.orchestra.run.smithi053.stdout:1343 write 0x1c79393 thru 0x1c7dd8e (0x49fc bytes) 2024-09-13T05:02:26.188 INFO:teuthology.orchestra.run.smithi053.stdout:1345 trunc from 0x2a7e5e5 to 0x8a48878 2024-09-13T05:02:26.194 INFO:teuthology.orchestra.run.smithi053.stdout:1346 write 0x28e12fd thru 0x28e8d85 (0x7a89 bytes) 2024-09-13T05:02:26.195 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.195 INFO:teuthology.orchestra.run.smithi053.stdout:1348 writesame 0x1578370 thru 0x1578ec7 data_size 0x8(0xb58 bytes) 2024-09-13T05:02:26.198 INFO:teuthology.orchestra.run.smithi053.stdout:1349 trunc from 0x8a48878 to 0x76cc987 2024-09-13T05:02:26.204 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.205 INFO:teuthology.orchestra.run.smithi053.stdout:1351 read 0x1b5cef6 thru 0x1b66e6f (0x9f7a bytes) 2024-09-13T05:02:26.205 INFO:teuthology.orchestra.run.smithi053.stdout:1352 read 0x371a252 thru 0x3721eee (0x7c9d bytes) 2024-09-13T05:02:26.205 INFO:teuthology.orchestra.run.smithi053.stdout:1354 read 0x46de01 thru 0x474c44 (0x6e44 bytes) 2024-09-13T05:02:26.205 INFO:teuthology.orchestra.run.smithi053.stdout:1355 read 0x214e585 thru 0x2156638 (0x80b4 bytes) 2024-09-13T05:02:26.206 INFO:teuthology.orchestra.run.smithi053.stdout:1356 read 0x33288d8 thru 0x3336d43 (0xe46c bytes) 2024-09-13T05:02:26.216 INFO:teuthology.orchestra.run.smithi053.stdout:1357 write 0xabd4e72 thru 0xabe3b6f (0xecfe bytes) 2024-09-13T05:02:26.227 INFO:teuthology.orchestra.run.smithi053.stdout:1358 writesame 0xda5c43f thru 0xda670b0 data_size 0x1(0xac72 bytes) 2024-09-13T05:02:26.227 INFO:teuthology.orchestra.run.smithi053.stdout:1359 writesame 0xaae61ab thru 0xaae8414 data_size 0x5(0x226a bytes) 2024-09-13T05:02:26.230 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.230 INFO:teuthology.orchestra.run.smithi053.stdout:1361 read 0x7b55305 thru 0x7b63de2 (0xeade bytes) 2024-09-13T05:02:26.230 INFO:teuthology.orchestra.run.smithi053.stdout:1363 read 0x5ab2205 thru 0x5ab9b05 (0x7901 bytes) 2024-09-13T05:02:26.230 INFO:teuthology.orchestra.run.smithi053.stdout:1364 read 0x4dc996f thru 0x4dd9946 (0xffd8 bytes) 2024-09-13T05:02:26.231 INFO:teuthology.orchestra.run.smithi053.stdout:1365 writesame 0x64e2cd3 thru 0x64f0344 data_size 0x1(0xd672 bytes) 2024-09-13T05:02:26.235 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:26.235+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:26.255 INFO:teuthology.orchestra.run.smithi053.stdout:1366 write 0xb3ca997 thru 0xb3d8d12 (0xe37c bytes) 2024-09-13T05:02:26.259 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.259 INFO:teuthology.orchestra.run.smithi053.stdout:1368 write 0xc0d0098 thru 0xc0e0003 (0xff6c bytes) 2024-09-13T05:02:26.271 INFO:teuthology.orchestra.run.smithi053.stdout:1370 trunc from 0xda670b1 to 0x4b67514 2024-09-13T05:02:26.293 INFO:teuthology.orchestra.run.smithi053.stdout:1371 write 0x9d2a053 thru 0x9d33e2b (0x9dd9 bytes) 2024-09-13T05:02:26.296 INFO:teuthology.orchestra.run.smithi053.stdout:1372 read 0x97951a9 thru 0x97a2ec3 (0xdd1b bytes) 2024-09-13T05:02:26.296 INFO:teuthology.orchestra.run.smithi053.stdout:1373 read 0xd3e3ec thru 0xd4859f (0xa1b4 bytes) 2024-09-13T05:02:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:1374 read 0x92877f thru 0x92a279 (0x1afb bytes) 2024-09-13T05:02:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.297 INFO:teuthology.orchestra.run.smithi053.stdout:1377 write 0x6625170 thru 0x6626847 (0x16d8 bytes) 2024-09-13T05:02:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:1378 punch from 0x800b512 to 0x80195c1, (0xe0af bytes) 2024-09-13T05:02:26.298 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.299 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:26.299 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:26.301 INFO:teuthology.orchestra.run.smithi053.stdout:1380 trunc from 0x9d33e2c to 0xc9e5678 2024-09-13T05:02:26.304 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:26.307 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.308 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:26.313 INFO:teuthology.orchestra.run.smithi053.stdout:1383 write 0xdb3d6f9 thru 0xdb3ec3d (0x1545 bytes) 2024-09-13T05:02:26.314 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:26.313+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:1386 read 0x6ebef1d thru 0x6ec2f52 (0x4036 bytes) 2024-09-13T05:02:26.316 INFO:teuthology.orchestra.run.smithi053.stdout:1387 writesame 0xc4169f3 thru 0xc41c8ae data_size 0x1(0x5ebc bytes) 2024-09-13T05:02:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:1390 read 0x58e1f4c thru 0x58efb91 (0xdc46 bytes) 2024-09-13T05:02:26.327 INFO:teuthology.orchestra.run.smithi053.stdout:1391 write 0x346aa67 thru 0x3473559 (0x8af3 bytes) 2024-09-13T05:02:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:1392 read 0x8a6d884 thru 0x8a76ab0 (0x922d bytes) 2024-09-13T05:02:26.329 INFO:teuthology.orchestra.run.smithi053.stdout:1394 read 0x56559a6 thru 0x5665690 (0xfceb bytes) 2024-09-13T05:02:26.330 INFO:teuthology.orchestra.run.smithi053.stdout:1395 trunc from 0xdb3ec3e to 0xdf216a4 2024-09-13T05:02:26.342 INFO:teuthology.orchestra.run.smithi053.stdout:1396 trunc from 0xdf216a4 to 0x7bc44c1 2024-09-13T05:02:26.348 INFO:teuthology.orchestra.run.smithi053.stdout:1398 read 0x617dc5a thru 0x6186ccd (0x9074 bytes) 2024-09-13T05:02:26.349 INFO:teuthology.orchestra.run.smithi053.stdout:1399 writesame 0x5358852 thru 0x536860c data_size 0xb(0xfdbb bytes) 2024-09-13T05:02:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:1401 punch from 0x1fcc49a to 0x1fcf424, (0x2f8a bytes) 2024-09-13T05:02:26.352 INFO:teuthology.orchestra.run.smithi053.stdout:1402 punch from 0x1d90e9e to 0x1da04c9, (0xf62b bytes) 2024-09-13T05:02:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:1403 writesame 0x912ccbc thru 0x913433b data_size 0x2(0x7680 bytes) 2024-09-13T05:02:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:1404 punch from 0x2d89425 to 0x2d9385d, (0xa438 bytes) 2024-09-13T05:02:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:1405 read 0x6819820 thru 0x6827380 (0xdb61 bytes) 2024-09-13T05:02:26.359 INFO:teuthology.orchestra.run.smithi053.stdout:1406 write 0x2a56b7e thru 0x2a58396 (0x1819 bytes) 2024-09-13T05:02:26.364 INFO:teuthology.orchestra.run.smithi053.stdout:1407 trunc from 0x913433c to 0x5de9cc7 2024-09-13T05:02:26.378 INFO:teuthology.orchestra.run.smithi053.stdout:1410 trunc from 0x5de9cc7 to 0x7cb4f15 2024-09-13T05:02:26.383 INFO:teuthology.orchestra.run.smithi053.stdout:1411 writesame 0x3b14fc1 thru 0x3b22fe6 data_size 0xd(0xe026 bytes) 2024-09-13T05:02:26.386 INFO:teuthology.orchestra.run.smithi053.stdout:1412 write 0x249d4e5 thru 0x24a7a1e (0xa53a bytes) 2024-09-13T05:02:26.388 INFO:teuthology.orchestra.run.smithi053.stdout:1414 write 0x6815c1e thru 0x6816cca (0x10ad bytes) 2024-09-13T05:02:26.389 INFO:teuthology.orchestra.run.smithi053.stdout:1415 read 0x627bbcf thru 0x6282443 (0x6875 bytes) 2024-09-13T05:02:26.396 INFO:teuthology.orchestra.run.smithi053.stdout:1416 writesame 0x927a447 thru 0x927d9f6 data_size 0x8(0x35b0 bytes) 2024-09-13T05:02:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.397 INFO:teuthology.orchestra.run.smithi053.stdout:1418 writesame 0x5b92aa1 thru 0x5b996dd data_size 0xb(0x6c3d bytes) 2024-09-13T05:02:26.398 INFO:teuthology.orchestra.run.smithi053.stdout:1419 write 0x571382b thru 0x5721827 (0xdffd bytes) 2024-09-13T05:02:26.400 INFO:teuthology.orchestra.run.smithi053.stdout:1420 read 0x340b5fb thru 0x340f5c9 (0x3fcf bytes) 2024-09-13T05:02:26.407 INFO:teuthology.orchestra.run.smithi053.stdout:1421 trunc from 0x927d9f7 to 0x2e38b67 2024-09-13T05:02:26.414 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:26.414+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.415 INFO:teuthology.orchestra.run.smithi053.stdout:1425 write 0x7b855 thru 0x87eff (0xc6ab bytes) 2024-09-13T05:02:26.429 INFO:teuthology.orchestra.run.smithi053.stdout:1426 trunc from 0x2e38b67 to 0xdac6106 2024-09-13T05:02:26.434 INFO:teuthology.orchestra.run.smithi053.stdout:1427 write 0xca81fce thru 0xca90728 (0xe75b bytes) 2024-09-13T05:02:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:1428 write 0xcd2cf85 thru 0xcd2e1bd (0x1239 bytes) 2024-09-13T05:02:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:1429 punch from 0xb063d70 to 0xb06fde5, (0xc075 bytes) 2024-09-13T05:02:26.440 INFO:teuthology.orchestra.run.smithi053.stdout:1430 write 0x37351fa thru 0x3735f90 (0xd97 bytes) 2024-09-13T05:02:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:1432 punch from 0x3ad2b9a to 0x3ae28ad, (0xfd13 bytes) 2024-09-13T05:02:26.441 INFO:teuthology.orchestra.run.smithi053.stdout:1433 write 0x5f9e7d0 thru 0x5fa5765 (0x6f96 bytes) 2024-09-13T05:02:26.442 INFO:teuthology.orchestra.run.smithi053.stdout:1434 write 0x5b85098 thru 0x5b8c207 (0x7170 bytes) 2024-09-13T05:02:26.445 INFO:teuthology.orchestra.run.smithi053.stdout:1435 trunc from 0xdac6106 to 0xbf64418 2024-09-13T05:02:26.446 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:26.446 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:26.452 INFO:teuthology.orchestra.run.smithi053.stdout:1436 read 0x1eabf96 thru 0x1eb20d3 (0x613e bytes) 2024-09-13T05:02:26.460 INFO:teuthology.orchestra.run.smithi053.stdout:1437 write 0xe54976f thru 0xe5501c3 (0x6a55 bytes) 2024-09-13T05:02:26.461 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:26.461 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:26.463 INFO:teuthology.orchestra.run.smithi053.stdout:1438 write 0x542a8c9 thru 0x54305c5 (0x5cfd bytes) 2024-09-13T05:02:26.464 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:26.465 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:26.465 INFO:teuthology.orchestra.run.smithi053.stdout:1439 trunc from 0xe5501c4 to 0xedf3238 2024-09-13T05:02:26.471 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:26.471 INFO:teuthology.orchestra.run.smithi053.stdout:1440 writesame 0x39dbe30 thru 0x39e0e8b data_size 0x4a(0x505c bytes) 2024-09-13T05:02:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:1443 read 0xc0dc44 thru 0xc12059 (0x4416 bytes) 2024-09-13T05:02:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:1444 read 0x908684b thru 0x9092975 (0xc12b bytes) 2024-09-13T05:02:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:26.472 INFO:teuthology.orchestra.run.smithi053.stdout:1446 punch from 0xe148054 to 0xe14cf98, (0x4f44 bytes) 2024-09-13T05:02:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:1448 trunc from 0xedf3238 to 0x66cc7f2 2024-09-13T05:02:26.482 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:26.482 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:26.492 INFO:teuthology.orchestra.run.smithi053.stdout:1449 clone 12 order 21 su 32768 sc 11 2024-09-13T05:02:26.500 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:26.500 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:26.515 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:26.515+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:26.516 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:26.516 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:26.533 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:26.533 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:26.549 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:26.550 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:26.566 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:26.566 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:26.615 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:26.615 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:26.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:26.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:26.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.615 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10 3.1c 3:3831950c:::rbd_directory:head [call rbd.dir_add_image in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 17.727351367000001, 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002690039, 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886008+0000", 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886006+0000", 2024-09-13T05:02:26.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886013+0000", 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0630000000000002e-06 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886021+0000", 2024-09-13T05:02:26.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3070000000000003e-06 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886030+0000", 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4909999999999994e-06 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886071+0000", 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:26.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.886076+0000", 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1679999999999997e-06 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887440+0000", 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001364142 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887556+0000", 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011576799999999999 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.887726+0000", 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017057500000000001 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888284+0000", 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00055794299999999998 2024-09-13T05:02:26.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888637+0000", 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00035292599999999999 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888674+0000", 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7416000000000002e-05 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:08.888698+0000", 2024-09-13T05:02:26.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3422999999999999e-05 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:1936 3.1c 3:3ac27fc6:::rbd_data.117cdd702390.0000000000000040:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.169390509999999, 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0065407260000000002, 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 1936 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443969+0000", 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443968+0000", 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443973+0000", 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1240000000000004e-06 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443984+0000", 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0796e-05 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.443990+0000", 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.939e-06 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.444026+0000", 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5697000000000001e-05 2024-09-13T05:02:26.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.444119+0000", 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2684999999999998e-05 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.445186+0000", 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001067274 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450407+0000", 2024-09-13T05:02:26.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0052206129999999998 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450454+0000", 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7132999999999997e-05 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450487+0000", 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2697999999999998e-05 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.450509+0000", 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2767000000000001e-05 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:2050 3.16 3:6bc3cb69:::rbd_data.117cdd702390.000000000000005a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 12.035930624000001, 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0036278289999999999, 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 2050 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577429+0000", 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577427+0000", 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577435+0000", 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0120000000000005e-06 2024-09-13T05:02:26.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577448+0000", 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3444999999999999e-05 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577456+0000", 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9619999999999993e-06 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577506+0000", 2024-09-13T05:02:26.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9738999999999999e-05 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.577611+0000", 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105074 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.578226+0000", 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061466299999999999 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.580639+0000", 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024128550000000002 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.580966+0000", 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00032758400000000001 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.581016+0000", 2024-09-13T05:02:26.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0105000000000002e-05 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:14.581056+0000", 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9962000000000001e-05 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9140 3.16 3:6bbed544:::rbd_data.117c6c62fa22.000000000000004e:head [truncate 212992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:26.635 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7026286729999995, 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0039799379999999997, 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9140 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:26.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910730+0000", 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910729+0000", 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910735+0000", 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2469999999999999e-06 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910746+0000", 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1048e-05 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910751+0000", 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2229999999999998e-06 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910792+0000", 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1136000000000001e-05 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.910798+0000", 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6119999999999998e-06 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913554+0000", 2024-09-13T05:02:26.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00275632 2024-09-13T05:02:26.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913600+0000", 2024-09-13T05:02:26.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6162000000000003e-05 2024-09-13T05:02:26.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 7", 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.913697+0000", 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6904999999999995e-05 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914317+0000", 2024-09-13T05:02:26.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062001999999999999 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914626+0000", 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00030938899999999997 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914674+0000", 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7221000000000001e-05 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.914710+0000", 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6718999999999999e-05 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9239 3.1d 3:ba6bf161:::rbd_data.117c6c62fa22.0000000000000151:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:26.647 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.6320019489999993, 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027959809999999999, 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9239 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981357+0000", 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981356+0000", 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981362+0000", 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7030000000000003e-06 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981373+0000", 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1063000000000001e-05 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981381+0000", 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2700000000000004e-06 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981418+0000", 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7049999999999999e-05 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.981500+0000", 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1798999999999994e-05 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.982080+0000", 2024-09-13T05:02:26.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058020599999999999 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.983823+0000", 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017424020000000001 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984107+0000", 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00028401700000000002 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984134+0000", 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7489000000000001e-05 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.984153+0000", 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9123999999999999e-05 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9278 3.1c 3:3c83d342:::rbd_data.117c6c62fa22.0000000000000043:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:26.653 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.6013026880000005, 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030035639999999998, 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9278 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:26.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012056+0000", 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012055+0000", 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012062+0000", 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6959999999999999e-06 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012076+0000", 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3994e-05 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012081+0000", 2024-09-13T05:02:26.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2499999999999997e-06 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012125+0000", 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3708000000000002e-05 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012208+0000", 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2834e-05 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012661+0000", 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045269800000000002 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014746+0000", 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020852079999999999 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014992+0000", 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024621199999999999 2024-09-13T05:02:26.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.015029+0000", 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6698e-05 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.015060+0000", 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1265e-05 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.659 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9394 3.1c 3:384f0210:::rbd_data.117c6c62fa22.00000000000000d5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.5218940270000001, 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002679439, 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9394 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091465+0000", 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091464+0000", 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091471+0000", 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6490000000000002e-06 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091487+0000", 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6529999999999999e-05 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091495+0000", 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2700000000000004e-06 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091552+0000", 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6740000000000002e-05 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091635+0000", 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2509000000000006e-05 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.091739+0000", 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000103915 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.092622+0000", 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00088372799999999997 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094066+0000", 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001444047 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094109+0000", 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2416999999999997e-05 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094145+0000", 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5819999999999999e-05 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9468 3.c 3:37c3b698:::rbd_data.117c6c62fa22.0000000000000162:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.4848503110000006, 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030936710000000001, 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9468 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128509+0000", 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128507+0000", 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128515+0000", 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4789999999999996e-06 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128533+0000", 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7227999999999998e-05 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128541+0000", 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128591+0000", 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9756999999999998e-05 2024-09-13T05:02:26.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.128715+0000", 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000123979 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.129064+0000", 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00034899799999999999 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131179+0000", 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002115268 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131516+0000", 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033735900000000002 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131564+0000", 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7469999999999998e-05 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.131602+0000", 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8892e-05 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9879 3.1d 3:b8dde86d:::rbd_data.117c6c62fa22.000000000000009e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:26.673 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.2565745689999996, 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0034236219999999999, 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9879 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356785+0000", 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356783+0000", 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356793+0000", 2024-09-13T05:02:26.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0233e-05 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356815+0000", 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2005999999999999e-05 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356825+0000", 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7540000000000001e-06 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.356892+0000", 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7088999999999994e-05 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.357053+0000", 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016157300000000001 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.357490+0000", 2024-09-13T05:02:26.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00043711299999999999 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360070+0000", 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025790990000000001 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360143+0000", 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3672999999999997e-05 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360175+0000", 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1690999999999999e-05 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.360208+0000", 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3260999999999999e-05 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10086 3.1c 3:385e9163:::rbd_data.117c6c62fa22.0000000000000003:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.679 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6358680119999995, 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027722110000000001, 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10086 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977491+0000", 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977490+0000", 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977496+0000", 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6699999999999997e-06 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977510+0000", 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4073000000000001e-05 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977518+0000", 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5270000000000001e-06 2024-09-13T05:02:26.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977573+0000", 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5096000000000002e-05 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977663+0000", 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.0365000000000006e-05 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977847+0000", 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00018395400000000001 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.979691+0000", 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018432190000000001 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980183+0000", 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049266399999999997 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980228+0000", 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4202000000000001e-05 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980263+0000", 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5822999999999999e-05 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.685 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10126 3.1c 3:3c309cef:::rbd_data.117c6c62fa22.000000000000002e:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6228597689999997, 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028308270000000002, 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10126 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990499+0000", 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990506+0000", 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8189999999999992e-06 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990516+0000", 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0033e-05 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990523+0000", 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4670000000000001e-06 2024-09-13T05:02:26.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990568+0000", 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4749000000000001e-05 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990650+0000", 2024-09-13T05:02:26.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2464000000000002e-05 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990806+0000", 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015617499999999999 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991958+0000", 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00115196 2024-09-13T05:02:26.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993253+0000", 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012944020000000001 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993296+0000", 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2957999999999998e-05 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993330+0000", 2024-09-13T05:02:26.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4650000000000002e-05 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10130 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6220155030000001, 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028473510000000001, 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10130 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991344+0000", 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991343+0000", 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991348+0000", 2024-09-13T05:02:26.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8300000000000001e-06 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991361+0000", 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2405000000000001e-05 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991373+0000", 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1815e-05 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991419+0000", 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6575e-05 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991592+0000", 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000172972 2024-09-13T05:02:26.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991755+0000", 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000162318 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.992830+0000", 2024-09-13T05:02:26.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010755809999999999 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994112+0000", 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012814409999999999 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994156+0000", 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3955000000000002e-05 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994191+0000", 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5475000000000002e-05 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10187 3.c 3:356439cd:::rbd_data.117c6c62fa22.00000000000000b2:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.6068538090000004, 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025044160000000002, 2024-09-13T05:02:26.699 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10187 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006505+0000", 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006504+0000", 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006510+0000", 2024-09-13T05:02:26.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5500000000000002e-06 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006519+0000", 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3619999999999995e-06 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006526+0000", 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6279999999999999e-06 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006577+0000", 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1347999999999997e-05 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006662+0000", 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5271000000000004e-05 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 3", 2024-09-13T05:02:26.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.006835+0000", 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000173148 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007618+0000", 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00078252200000000001 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008945+0000", 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001326767 2024-09-13T05:02:26.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008976+0000", 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1466e-05 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.009010+0000", 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3590999999999997e-05 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10773 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 319488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0743196719999997, 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026201689999999999, 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10773 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539039+0000", 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539041+0000", 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.199e-06 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539047+0000", 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.57e-06 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539051+0000", 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6360000000000002e-06 2024-09-13T05:02:26.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541497+0000", 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024458610000000001 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541571+0000", 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4208999999999994e-05 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541660+0000", 2024-09-13T05:02:26.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8208000000000007e-05 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10774 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 327680~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0743038189999998, 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027446580000000001, 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.710 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10774 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539055+0000", 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539060+0000", 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8080000000000004e-06 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539066+0000", 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.784e-06 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539070+0000", 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7949999999999998e-06 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541689+0000", 2024-09-13T05:02:26.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026189709999999999 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541743+0000", 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4032e-05 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541800+0000", 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6638999999999998e-05 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.714 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10775 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 339968~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539075+0000", 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0742843300000002, 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002929737, 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10775 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539075+0000", 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539075+0000", 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539074+0000", 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539077+0000", 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539087+0000", 2024-09-13T05:02:26.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0696999999999999e-05 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539093+0000", 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.468e-06 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541834+0000", 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027409029999999998 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541909+0000", 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5204999999999996e-05 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542005+0000", 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5662000000000007e-05 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10776 3.5 3:a62ab9ca:::rbd_data.117c6c62fa22.0000000000000032:9 [read 352256~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:26.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0742619869999999, 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030923719999999999, 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10776 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:26.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539097+0000", 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539099+0000", 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3149999999999999e-06 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539110+0000", 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0540000000000001e-05 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.539115+0000", 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3449999999999996e-06 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542030+0000", 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002914626 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542103+0000", 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3780000000000002e-05 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542190+0000", 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6161000000000004e-05 2024-09-13T05:02:26.723 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10825 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:9 [read 315392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.540367+0000", 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.072992578, 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025250200000000002, 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10825 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540367+0000", 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540367+0000", 2024-09-13T05:02:26.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540366+0000", 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540369+0000", 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540376+0000", 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1840000000000002e-06 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540381+0000", 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0370000000000002e-06 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542791+0000", 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024095539999999999 2024-09-13T05:02:26.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542838+0000", 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7141000000000001e-05 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542892+0000", 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3899e-05 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10826 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:9 [read 327680~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.540399+0000", 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0729606240000003, 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027129720000000001, 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10826 2024-09-13T05:02:26.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540399+0000", 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540399+0000", 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540398+0000", 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.540400+0000", 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541020+0000", 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00061979099999999996 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541029+0000", 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3839999999999997e-06 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.542916+0000", 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001887021 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543004+0000", 2024-09-13T05:02:26.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8658000000000004e-05 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543111+0000", 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000107142 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10829 3.5 3:a6f8a8fc:::rbd_data.117c6c62fa22.000000000000002c:9 [read 360448~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0722766769999996, 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002601819, 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10829 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:26.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541082+0000", 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:26.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541085+0000", 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.537e-06 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541091+0000", 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3289999999999999e-06 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:26.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.541096+0000", 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9490000000000004e-06 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543497+0000", 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024008509999999999 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543584+0000", 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7336000000000005e-05 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:26.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.543684+0000", 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100234 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:26.738 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:26.740 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:26.740 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:26.739+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:26.841 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:26.840+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:26.842 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:26.842 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:26.850 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:26.943 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:26.942+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:26.943 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:02:26.955 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:26.955 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:26.955 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:26.955 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:26.955 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:26.965 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:27.022 INFO:teuthology.orchestra.run.smithi053.stdout:flattening image image_client.0-clone11 2024-09-13T05:02:27.043 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:27.043+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:27.071 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8383 3.10 3:0d0d9ed8:::rbd_data.117c6c62fa22.000000000000017a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.247621+0000", 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.8221625429999992, 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022406259999999999, 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.072 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8383 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247621+0000", 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247621+0000", 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247620+0000", 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247625+0000", 2024-09-13T05:02:27.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7929999999999997e-06 2024-09-13T05:02:27.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247636+0000", 2024-09-13T05:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0541e-05 2024-09-13T05:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247641+0000", 2024-09-13T05:02:27.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2750000000000003e-06 2024-09-13T05:02:27.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247685+0000", 2024-09-13T05:02:27.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3639e-05 2024-09-13T05:02:27.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247765+0000", 2024-09-13T05:02:27.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0663999999999999e-05 2024-09-13T05:02:27.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.247862+0000", 2024-09-13T05:02:27.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6521000000000006e-05 2024-09-13T05:02:27.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.248559+0000", 2024-09-13T05:02:27.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00069733800000000004 2024-09-13T05:02:27.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.249794+0000", 2024-09-13T05:02:27.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001235002 2024-09-13T05:02:27.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.249826+0000", 2024-09-13T05:02:27.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2081e-05 2024-09-13T05:02:27.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.249862+0000", 2024-09-13T05:02:27.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5670000000000002e-05 2024-09-13T05:02:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.084 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8615 3.10 3:0cc2e45c:::rbd_data.117c6c62fa22.000000000000003a:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:27.084 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.404628+0000", 2024-09-13T05:02:27.084 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.6651556280000008, 2024-09-13T05:02:27.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0044752689999999996, 2024-09-13T05:02:27.084 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.084 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.085 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.085 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8615 2024-09-13T05:02:27.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.085 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404628+0000", 2024-09-13T05:02:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404628+0000", 2024-09-13T05:02:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404626+0000", 2024-09-13T05:02:27.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999976 2024-09-13T05:02:27.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404637+0000", 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0669999999999999e-05 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404655+0000", 2024-09-13T05:02:27.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8067000000000002e-05 2024-09-13T05:02:27.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404664+0000", 2024-09-13T05:02:27.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.329e-06 2024-09-13T05:02:27.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404712+0000", 2024-09-13T05:02:27.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7691000000000001e-05 2024-09-13T05:02:27.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.404818+0000", 2024-09-13T05:02:27.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010636099999999999 2024-09-13T05:02:27.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.405694+0000", 2024-09-13T05:02:27.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00087614500000000005 2024-09-13T05:02:27.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.408785+0000", 2024-09-13T05:02:27.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030904639999999998 2024-09-13T05:02:27.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409049+0000", 2024-09-13T05:02:27.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00026435 2024-09-13T05:02:27.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409081+0000", 2024-09-13T05:02:27.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2152e-05 2024-09-13T05:02:27.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.409103+0000", 2024-09-13T05:02:27.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2289999999999998e-05 2024-09-13T05:02:27.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8710 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [set-alloc-hint object_size 524288 write_size 524288,write 217088~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:27.098 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.458384+0000", 2024-09-13T05:02:27.098 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.6113997530000006, 2024-09-13T05:02:27.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026806640000000001, 2024-09-13T05:02:27.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.099 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.099 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8710 2024-09-13T05:02:27.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.099 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458384+0000", 2024-09-13T05:02:27.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458384+0000", 2024-09-13T05:02:27.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458382+0000", 2024-09-13T05:02:27.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:27.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458395+0000", 2024-09-13T05:02:27.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2634e-05 2024-09-13T05:02:27.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458411+0000", 2024-09-13T05:02:27.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6489000000000001e-05 2024-09-13T05:02:27.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458420+0000", 2024-09-13T05:02:27.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7260000000000008e-06 2024-09-13T05:02:27.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458474+0000", 2024-09-13T05:02:27.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3769e-05 2024-09-13T05:02:27.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458557+0000", 2024-09-13T05:02:27.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2984000000000004e-05 2024-09-13T05:02:27.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.458699+0000", 2024-09-13T05:02:27.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014173500000000001 2024-09-13T05:02:27.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.459521+0000", 2024-09-13T05:02:27.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00082235700000000001 2024-09-13T05:02:27.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.460987+0000", 2024-09-13T05:02:27.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001466237 2024-09-13T05:02:27.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.461029+0000", 2024-09-13T05:02:27.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2188000000000003e-05 2024-09-13T05:02:27.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.461065+0000", 2024-09-13T05:02:27.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5321000000000003e-05 2024-09-13T05:02:27.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.111 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.112 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8882 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [write 147456~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:27.112 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.560284+0000", 2024-09-13T05:02:27.112 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.5094999340000008, 2024-09-13T05:02:27.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023403019999999998, 2024-09-13T05:02:27.112 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.113 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.113 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8882 2024-09-13T05:02:27.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.114 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560284+0000", 2024-09-13T05:02:27.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560284+0000", 2024-09-13T05:02:27.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560283+0000", 2024-09-13T05:02:27.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560292+0000", 2024-09-13T05:02:27.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1260000000000004e-06 2024-09-13T05:02:27.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560303+0000", 2024-09-13T05:02:27.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.041e-05 2024-09-13T05:02:27.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560308+0000", 2024-09-13T05:02:27.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3750000000000002e-06 2024-09-13T05:02:27.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560346+0000", 2024-09-13T05:02:27.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8031999999999998e-05 2024-09-13T05:02:27.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560425+0000", 2024-09-13T05:02:27.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8768000000000002e-05 2024-09-13T05:02:27.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.560543+0000", 2024-09-13T05:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0001182 2024-09-13T05:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.561304+0000", 2024-09-13T05:02:27.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00076059800000000002 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.562548+0000", 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001243971 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.562592+0000", 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4764e-05 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.562624+0000", 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1850000000000002e-05 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.124 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:8970 3.a 3:547253cd:::rbd_data.117c6c62fa22.000000000000002a:head [set-alloc-hint object_size 524288 write_size 524288,write 446464~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:17.808539+0000", 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.2612447119999999, 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028653709999999998, 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 8970 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808539+0000", 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808539+0000", 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808537+0000", 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808552+0000", 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4345000000000001e-05 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808566+0000", 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4552000000000001e-05 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808574+0000", 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7069999999999992e-06 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808620+0000", 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6023999999999998e-05 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808746+0000", 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012644800000000001 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.808882+0000", 2024-09-13T05:02:27.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000135747 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.809502+0000", 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00062012700000000005 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.811319+0000", 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0018165919999999999 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.811367+0000", 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8491e-05 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:17.811404+0000", 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7092999999999999e-05 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9276 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.012014+0000", 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 9.0577699050000007, 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029183199999999999, 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9276 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012014+0000", 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012014+0000", 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012013+0000", 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012018+0000", 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9189999999999998e-06 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012028+0000", 2024-09-13T05:02:27.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0199e-05 2024-09-13T05:02:27.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012034+0000", 2024-09-13T05:02:27.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:02:27.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012070+0000", 2024-09-13T05:02:27.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6396999999999997e-05 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012145+0000", 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4802000000000007e-05 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.012564+0000", 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00041880599999999997 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014635+0000", 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020714489999999999 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014884+0000", 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00024929299999999999 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014912+0000", 2024-09-13T05:02:27.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8209e-05 2024-09-13T05:02:27.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.014932+0000", 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9779000000000002e-05 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9401 3.10 3:0902317e:::rbd_data.117c6c62fa22.00000000000000e1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.093414+0000", 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.9763696569999993, 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024931250000000001, 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9401 2024-09-13T05:02:27.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093414+0000", 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093414+0000", 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093413+0000", 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093422+0000", 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7339999999999997e-06 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093435+0000", 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3278000000000001e-05 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093443+0000", 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4599999999999997e-06 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093489+0000", 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6377e-05 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093568+0000", 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9433999999999999e-05 2024-09-13T05:02:27.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.093663+0000", 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4806e-05 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094246+0000", 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00058259100000000003 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095828+0000", 2024-09-13T05:02:27.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001582186 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095873+0000", 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5460000000000002e-05 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095907+0000", 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3899999999999997e-05 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9410 3.10 3:0d7acefb:::rbd_data.117c6c62fa22.000000000000010a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.094592+0000", 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.9751923130000009, 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023194359999999998, 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9410 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.148 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094592+0000", 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094592+0000", 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094591+0000", 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094596+0000", 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.288e-06 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094608+0000", 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2378000000000001e-05 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094619+0000", 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0706999999999999e-05 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094739+0000", 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000120098 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094823+0000", 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3424000000000006e-05 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.094899+0000", 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6810000000000005e-05 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.095401+0000", 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00050169600000000004 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.096843+0000", 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001441583 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.096885+0000", 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2052999999999999e-05 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.096911+0000", 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6256999999999999e-05 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9515 3.a 3:5630f440:::rbd_data.117c6c62fa22.000000000000006b:head [set-alloc-hint object_size 524288 write_size 524288,write 186907~1509] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.9092274279999994, 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003723638, 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9515 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.154 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160556+0000", 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160561+0000", 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1880000000000001e-06 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160572+0000", 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0845999999999999e-05 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160578+0000", 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7050000000000002e-06 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160609+0000", 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0664000000000003e-05 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160683+0000", 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4041000000000005e-05 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.160756+0000", 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2745999999999998e-05 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.161214+0000", 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00045812700000000001 2024-09-13T05:02:27.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.164194+0000", 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00298008 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.164239+0000", 2024-09-13T05:02:27.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5402999999999999e-05 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.164280+0000", 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0717999999999999e-05 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9786 3.a 3:54deefe6:::rbd_data.117c6c62fa22.0000000000000041:head [set-alloc-hint object_size 524288 write_size 524288,write 172032~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.297975+0000", 2024-09-13T05:02:27.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7718092040000002, 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0037219729999999999, 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9786 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297975+0000", 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297975+0000", 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297973+0000", 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.297991+0000", 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8079000000000001e-05 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298011+0000", 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0489e-05 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298021+0000", 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6509999999999996e-06 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298067+0000", 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5596999999999997e-05 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298138+0000", 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0932000000000005e-05 2024-09-13T05:02:27.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.298277+0000", 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013981200000000001 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.299096+0000", 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00081815299999999996 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.301606+0000", 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025103830000000001 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.301658+0000", 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1974000000000001e-05 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.301697+0000", 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8661999999999997e-05 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.166 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9878 3.a 3:542814f5:::rbd_data.117c6c62fa22.000000000000009d:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.355033+0000", 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7147507599999994, 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0070892680000000001, 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9878 2024-09-13T05:02:27.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355033+0000", 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355033+0000", 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355031+0000", 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355042+0000", 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0468e-05 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355058+0000", 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6905000000000002e-05 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355066+0000", 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8690000000000004e-06 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355121+0000", 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4586000000000002e-05 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355197+0000", 2024-09-13T05:02:27.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6236999999999995e-05 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.355603+0000", 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000405876 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.358219+0000", 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026158959999999999 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362058+0000", 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00383928 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362097+0000", 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8685e-05 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362122+0000", 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5435999999999998e-05 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:9882 3.10 3:0d194f86:::rbd_data.117c6c62fa22.00000000000000a2:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.362188+0000", 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7075961080000006, 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029458700000000002, 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 9882 2024-09-13T05:02:27.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362188+0000", 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362188+0000", 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362186+0000", 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362193+0000", 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1729999999999998e-06 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362206+0000", 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2116000000000001e-05 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362211+0000", 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9860000000000002e-06 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362253+0000", 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.269e-05 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362393+0000", 2024-09-13T05:02:27.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000139951 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.362889+0000", 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00049619300000000002 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.364752+0000", 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001862608 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.365043+0000", 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029077500000000001 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.365093+0000", 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0680000000000003e-05 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.365134+0000", 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0117999999999998e-05 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.178 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop(client.4476.0:10084 3.1e e29/19 3:7bb5a129:::rbd_data.117c6c62fa22.0000000000000001:head v 29'51, mlcod=28'48)", 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.977835+0000", 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.0919484710000003, 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002452503, 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977835+0000", 2024-09-13T05:02:27.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977835+0000", 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977834+0000", 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977845+0000", 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0953e-05 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977862+0000", 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6376e-05 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977869+0000", 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.926e-06 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977947+0000", 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7533000000000005e-05 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.977971+0000", 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4165999999999999e-05 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980254+0000", 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002283205 2024-09-13T05:02:27.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980288+0000", 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3469000000000001e-05 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10093 3.10 3:0efb4843:::rbd_data.117c6c62fa22.000000000000000a:head [stat,delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.980322+0000", 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.0894617590000006, 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029339129999999998, 2024-09-13T05:02:27.183 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10093 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980322+0000", 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980322+0000", 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980321+0000", 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980326+0000", 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8980000000000001e-06 2024-09-13T05:02:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980339+0000", 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2782000000000001e-05 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980346+0000", 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0690000000000004e-06 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980396+0000", 2024-09-13T05:02:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9403000000000001e-05 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980478+0000", 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1834000000000003e-05 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.980627+0000", 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00014940299999999999 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.982097+0000", 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001469896 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.983207+0000", 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011104470000000001 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.983233+0000", 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5383000000000001e-05 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.983256+0000", 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3308000000000001e-05 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10122 3.a 3:547253cd:::rbd_data.117c6c62fa22.000000000000002a:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.989485+0000", 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.0802985419999995, 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022556690000000001, 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.189 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10122 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989485+0000", 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989485+0000", 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989483+0000", 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989492+0000", 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6889999999999993e-06 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989512+0000", 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0024999999999999e-05 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989521+0000", 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2879999999999998e-06 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989557+0000", 2024-09-13T05:02:27.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5500999999999998e-05 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989621+0000", 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4333999999999998e-05 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.989740+0000", 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00011927800000000001 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.990479+0000", 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00073908000000000003 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991647+0000", 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001167916 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991693+0000", 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5414000000000003e-05 2024-09-13T05:02:27.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.991741+0000", 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8143999999999998e-05 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10138 3.10 3:0cc2e45c:::rbd_data.117c6c62fa22.000000000000003a:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:18.993508+0000", 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.0762757609999998, 2024-09-13T05:02:27.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022845640000000002, 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10138 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993508+0000", 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993508+0000", 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993507+0000", 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993513+0000", 2024-09-13T05:02:27.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5470000000000002e-06 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993528+0000", 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4773999999999999e-05 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993536+0000", 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7090000000000002e-06 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993574+0000", 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8152000000000003e-05 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993633+0000", 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9630000000000003e-05 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.993778+0000", 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000144771 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.994625+0000", 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00084671200000000001 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.995706+0000", 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001080777 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.995753+0000", 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-05 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:18.995793+0000", 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9158e-05 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:10192 3.10 3:0c2bfc09:::rbd_data.117c6c62fa22.0000000000000106:head [delete] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-13T05:02:27.201 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:19.007406+0000", 2024-09-13T05:02:27.204 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.0623781690000005, 2024-09-13T05:02:27.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0027390090000000001, 2024-09-13T05:02:27.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 10192 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007406+0000", 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007406+0000", 2024-09-13T05:02:27.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007405+0000", 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007414+0000", 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9509999999999995e-06 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007428+0000", 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4450000000000001e-05 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007445+0000", 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.6739e-05 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007498+0000", 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2859000000000003e-05 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007585+0000", 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7253e-05 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.007755+0000", 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000169772 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.008565+0000", 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080988900000000005 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010064+0000", 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014997389999999999 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010111+0000", 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6657999999999999e-05 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:19.010145+0000", 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3631999999999999e-05 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:20060 3.10 3:085258df:::rbd_object_map.117c7aa26b99:head [call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:22.968786+0000", 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.1009980270000002, 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002612376, 2024-09-13T05:02:27.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 20060 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968786+0000", 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968786+0000", 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968784+0000", 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968793+0000", 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8340000000000005e-06 2024-09-13T05:02:27.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968808+0000", 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5037e-05 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968815+0000", 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6019999999999996e-06 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968881+0000", 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6126999999999999e-05 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op must wait for map", 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.968887+0000", 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1070000000000001e-06 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970175+0000", 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001287814 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970242+0000", 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7735000000000001e-05 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970347+0000", 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000104975 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.970870+0000", 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052217500000000005 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.971351+0000", 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00048108899999999997 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.971378+0000", 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7727999999999999e-05 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:22.971398+0000", 2024-09-13T05:02:27.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.982e-05 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:20740 3.a 3:57d1c149:::rbd_data.117c7aa26b99.0000000000000018:head [set-alloc-hint object_size 4194304 write_size 4194304,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e32)", 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:23.552019+0000", 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.5177645549999998, 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.010494427000000001, 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 20740 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552019+0000", 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552019+0000", 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552018+0000", 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552023+0000", 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2640000000000002e-06 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552031+0000", 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2979999999999992e-06 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552036+0000", 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7779999999999998e-06 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552071+0000", 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4959999999999997e-05 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.552152+0000", 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1033999999999997e-05 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.553947+0000", 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00179474 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562360+0000", 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0084130699999999999 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562408+0000", 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8727000000000003e-05 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562463+0000", 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5041999999999999e-05 2024-09-13T05:02:27.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562514+0000", 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0306999999999998e-05 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_repop_reply(client.4476.0:20740 3.a e32/19 ondisk, result = 0)", 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:23.559878+0000", 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.5099053850000002, 2024-09-13T05:02:27.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026439039999999999, 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.559878+0000", 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.559878+0000", 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.559877+0000", 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.559883+0000", 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4229999999999996e-06 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.559895+0000", 2024-09-13T05:02:27.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2125e-05 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.559901+0000", 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5150000000000003e-06 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562390+0000", 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002488526 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562401+0000", 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0856000000000001e-05 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:23.562522+0000", 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000121583 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.227 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:27.228 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-13T05:02:27.229 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:27.228+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:27.229 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:27.330 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:27.329+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:27.340 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:27.340 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:27.348 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:27.430 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:27.430+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:27.494 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:27.494 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:27.494 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:27.494 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:27.494 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:27.502 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:27.546 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:27.546+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.607 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3765657130000002, 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:02:27.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:02:27.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.375637604, 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:02:27.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:02:27.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:02:27.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3745391950000001, 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:02:27.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.368479234, 2024-09-13T05:02:27.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:02:27.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:02:27.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.627 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3684702390000001, 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:02:27.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3684618679999998, 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:02:27.632 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:02:27.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:02:27.636 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3684518470000002, 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.637 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.641 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.368436124, 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:02:27.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.646 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.366533955, 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:02:27.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.366525996, 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.651 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:27.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:02:27.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3665183079999998, 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:02:27.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3665076159999998, 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:02:27.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:02:27.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.668 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.668 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.668 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3664993750000001, 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:02:27.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:02:27.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.673 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3664757139999999, 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:27.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:02:27.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3648332550000002, 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.678 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:02:27.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:02:27.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3648250219999998, 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:27.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.687 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3646022929999999, 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:02:27.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3645897439999999, 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:27.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:02:27.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3625505179999999, 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:02:27.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:02:27.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.3625407389999999, 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.703 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:27.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:27.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:27.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:27.708 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:27.710 INFO:teuthology.orchestra.run.smithi053.stdout:1450 read 0x2de8431 thru 0x2ded838 (0x5408 bytes) 2024-09-13T05:02:27.710 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.710 INFO:teuthology.orchestra.run.smithi053.stdout:1452 read 0x16b6d07 thru 0x16c4629 (0xd923 bytes) 2024-09-13T05:02:27.710 INFO:teuthology.orchestra.run.smithi053.stdout:1453 trunc from 0x66cc7f2 to 0xa9f9a49 2024-09-13T05:02:27.710 INFO:teuthology.orchestra.run.smithi053.stdout:1456 writesame 0x7a1fded thru 0x7a2b798 data_size 0x66(0xb9ac bytes) 2024-09-13T05:02:27.711 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:27.709+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:27.711 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:27.714 INFO:teuthology.orchestra.run.smithi053.stdout:1457 write 0x78d39ca thru 0x78e21b5 (0xe7ec bytes) 2024-09-13T05:02:27.719 INFO:teuthology.orchestra.run.smithi053.stdout:1458 read 0x9a62a2c thru 0x9a6a4c1 (0x7a96 bytes) 2024-09-13T05:02:27.719 INFO:teuthology.orchestra.run.smithi053.stdout:1460 punch from 0x6471942 to 0x6477f2d, (0x65eb bytes) 2024-09-13T05:02:27.719 INFO:teuthology.orchestra.run.smithi053.stdout:1461 read 0x7588e95 thru 0x75939ce (0xab3a bytes) 2024-09-13T05:02:27.727 INFO:teuthology.orchestra.run.smithi053.stdout:1462 write 0xd8e1418 thru 0xd8e5535 (0x411e bytes) 2024-09-13T05:02:27.730 INFO:teuthology.orchestra.run.smithi053.stdout:1463 read 0x3786e9f thru 0x378b834 (0x4996 bytes) 2024-09-13T05:02:27.730 INFO:teuthology.orchestra.run.smithi053.stdout:1464 writesame 0xb2f419c thru 0xb30343a data_size 0x13(0xf29f bytes) 2024-09-13T05:02:27.735 INFO:teuthology.orchestra.run.smithi053.stdout:1468 read 0x13abc1c thru 0x13b4844 (0x8c29 bytes) 2024-09-13T05:02:27.736 INFO:teuthology.orchestra.run.smithi053.stdout:1470 write 0x1ca556d thru 0x1cabd14 (0x67a8 bytes) 2024-09-13T05:02:27.751 INFO:teuthology.orchestra.run.smithi053.stdout:1471 writesame 0xb863602 thru 0xb86d912 data_size 0x21(0xa311 bytes) 2024-09-13T05:02:27.754 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.755 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.755 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.766 INFO:teuthology.orchestra.run.smithi053.stdout:1478 trunc from 0xd8e5536 to 0x3c9e487 2024-09-13T05:02:27.810 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:27.809+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:27.811 INFO:teuthology.orchestra.run.smithi053.stdout:1480 read 0x1404351 thru 0x140b619 (0x72c9 bytes) 2024-09-13T05:02:27.827 INFO:teuthology.orchestra.run.smithi053.stdout:1481 writesame 0xcbc9e0d thru 0xcbccc74 data_size 0x42(0x2e68 bytes) 2024-09-13T05:02:27.827 INFO:teuthology.orchestra.run.smithi053.stdout:1482 punch from 0x96e7819 to 0x96ec762, (0x4f49 bytes) 2024-09-13T05:02:27.827 INFO:teuthology.orchestra.run.smithi053.stdout:1483 punch from 0x5a649d0 to 0x5a6765d, (0x2c8d bytes) 2024-09-13T05:02:27.828 INFO:teuthology.orchestra.run.smithi053.stdout:1484 write 0x6d8af6c thru 0x6d97763 (0xc7f8 bytes) 2024-09-13T05:02:27.830 INFO:teuthology.orchestra.run.smithi053.stdout:1486 punch from 0x8ee8cf6 to 0x8ee9b01, (0xe0b bytes) 2024-09-13T05:02:27.831 INFO:teuthology.orchestra.run.smithi053.stdout:1487 trunc from 0xcbccc75 to 0xd942186 2024-09-13T05:02:27.836 INFO:teuthology.orchestra.run.smithi053.stdout:1488 read 0x7c1395a thru 0x7c1acb2 (0x7359 bytes) 2024-09-13T05:02:27.836 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.836 INFO:teuthology.orchestra.run.smithi053.stdout:1491 writesame 0xab4c824 thru 0xab58baf data_size 0xa(0xc38c bytes) 2024-09-13T05:02:27.841 INFO:teuthology.orchestra.run.smithi053.stdout:1492 writesame 0x997d6c3 thru 0x9980a8a data_size 0x4(0x33c8 bytes) 2024-09-13T05:02:27.845 INFO:teuthology.orchestra.run.smithi053.stdout:1493 punch from 0xbfd07ee to 0xbfd110f, (0x921 bytes) 2024-09-13T05:02:27.845 INFO:teuthology.orchestra.run.smithi053.stdout:1494 writesame 0x65a34c0 thru 0x65a3e4e data_size 0x1(0x98f bytes) 2024-09-13T05:02:27.847 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.847 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.847 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:27.847 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:27.847 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:27.848 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:27.848 INFO:teuthology.orchestra.run.smithi053.stdout:1499 write 0x6f9e00e thru 0x6fad187 (0xf17a bytes) 2024-09-13T05:02:27.851 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:27.860 INFO:teuthology.orchestra.run.smithi053.stdout:1502 trunc from 0xd942186 to 0x5c4118f 2024-09-13T05:02:27.879 INFO:teuthology.orchestra.run.smithi053.stdout:1503 write 0x9065491 thru 0x9069aa9 (0x4619 bytes) 2024-09-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi053.stdout:1504 punch from 0x610553d to 0x6107b9a, (0x265d bytes) 2024-09-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi053.stdout:1505 punch from 0x8dcb94b to 0x8dd9c82, (0xe337 bytes) 2024-09-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.884 INFO:teuthology.orchestra.run.smithi053.stdout:1509 writesame 0x1049e29 thru 0x10583fe data_size 0x1f(0xe5d6 bytes) 2024-09-13T05:02:27.903 INFO:teuthology.orchestra.run.smithi053.stdout:1510 write 0x7584674 thru 0x7589dca (0x5757 bytes) 2024-09-13T05:02:27.907 INFO:teuthology.orchestra.run.smithi053.stdout:1511 punch from 0x6e907a4 to 0x6e96889, (0x60e5 bytes) 2024-09-13T05:02:27.907 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.907 INFO:teuthology.orchestra.run.smithi053.stdout:1514 punch from 0x1ade2d9 to 0x1ae485c, (0x6583 bytes) 2024-09-13T05:02:27.907 INFO:teuthology.orchestra.run.smithi053.stdout:1515 write 0x2f47bc5 thru 0x2f513f0 (0x982c bytes) 2024-09-13T05:02:27.908 INFO:teuthology.orchestra.run.smithi053.stdout:1517 read 0x3e4acc thru 0x3e5a54 (0xf89 bytes) 2024-09-13T05:02:27.909 INFO:teuthology.orchestra.run.smithi053.stdout:1518 write 0x8a4dd4f thru 0x8a51f95 (0x4247 bytes) 2024-09-13T05:02:27.911 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:27.910+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:27.912 INFO:teuthology.orchestra.run.smithi053.stdout:1519 read 0x3a4323 thru 0x3a7ac2 (0x37a0 bytes) 2024-09-13T05:02:27.912 INFO:teuthology.orchestra.run.smithi053.stdout:1520 read 0x62eb3db thru 0x62f5a81 (0xa6a7 bytes) 2024-09-13T05:02:27.913 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.913 INFO:teuthology.orchestra.run.smithi053.stdout:1522 write 0x7d33fe0 thru 0x7d40ce7 (0xcd08 bytes) 2024-09-13T05:02:27.917 INFO:teuthology.orchestra.run.smithi053.stdout:1523 punch from 0x760f7e3 to 0x761d632, (0xde4f bytes) 2024-09-13T05:02:27.922 INFO:teuthology.orchestra.run.smithi053.stdout:1525 trunc from 0x9069aaa to 0xd2f7a91 2024-09-13T05:02:27.926 INFO:teuthology.orchestra.run.smithi053.stdout:1526 write 0x6456951 thru 0x6456b3b (0x1eb bytes) 2024-09-13T05:02:27.929 INFO:teuthology.orchestra.run.smithi053.stdout:1527 write 0xb3ea671 thru 0xb3fa375 (0xfd05 bytes) 2024-09-13T05:02:27.933 INFO:teuthology.orchestra.run.smithi053.stdout:1529 writesame 0xb0719d9 thru 0xb07ba40 data_size 0x4(0xa068 bytes) 2024-09-13T05:02:27.939 INFO:teuthology.orchestra.run.smithi053.stdout:1530 read 0x7c028f9 thru 0x7c08a28 (0x6130 bytes) 2024-09-13T05:02:27.939 INFO:teuthology.orchestra.run.smithi053.stdout:1531 write 0xa415e56 thru 0xa4242ea (0xe495 bytes) 2024-09-13T05:02:27.948 INFO:teuthology.orchestra.run.smithi053.stdout:1532 trunc from 0xd2f7a91 to 0x9aad293 2024-09-13T05:02:27.958 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.959 INFO:teuthology.orchestra.run.smithi053.stdout:1534 writesame 0x408835c thru 0x408cad5 data_size 0x7(0x477a bytes) 2024-09-13T05:02:27.960 INFO:teuthology.orchestra.run.smithi053.stdout:1535 write 0x7b5d9ba thru 0x7b6982f (0xbe76 bytes) 2024-09-13T05:02:27.967 INFO:teuthology.orchestra.run.smithi053.stdout:1536 trunc from 0x9aad293 to 0xc2a156b 2024-09-13T05:02:27.971 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:27.971 INFO:teuthology.orchestra.run.smithi053.stdout:1538 write 0x574e568 thru 0x5754bba (0x6653 bytes) 2024-09-13T05:02:27.975 INFO:teuthology.orchestra.run.smithi053.stdout:1539 read 0x6ff465c thru 0x6ff5fef (0x1994 bytes) 2024-09-13T05:02:27.975 INFO:teuthology.orchestra.run.smithi053.stdout:1540 read 0x887b86c thru 0x888765c (0xbdf1 bytes) 2024-09-13T05:02:27.975 INFO:teuthology.orchestra.run.smithi053.stdout:1541 punch from 0x132bf43 to 0x1337968, (0xba25 bytes) 2024-09-13T05:02:27.976 INFO:teuthology.orchestra.run.smithi053.stdout:1542 clone 13 order 24 su 1048576 sc 2 2024-09-13T05:02:27.983 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:27.984 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:27.984 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:27.984 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:27.984 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:27.993 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:28.034 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:28.033+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:28.112 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:28.112+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:28.141 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:28.141 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:28.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:28.141 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:28.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8280 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149355+0000", 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990739817, 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019844166, 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8280 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149355+0000", 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149355+0000", 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149355+0000", 2024-09-13T05:02:28.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149357+0000", 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.294e-06 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149363+0000", 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.19e-06 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149370+0000", 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6370000000000002e-06 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169189+0000", 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019819099999999999 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169199+0000", 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0349999999999999e-05 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8281 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149375+0000", 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990720137, 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019860036000000001, 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8281 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149375+0000", 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149375+0000", 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149374+0000", 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149377+0000", 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6340000000000001e-06 2024-09-13T05:02:28.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149383+0000", 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5289999999999999e-06 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149387+0000", 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3089999999999997e-06 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169228+0000", 2024-09-13T05:02:28.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019840703000000001 2024-09-13T05:02:28.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169235+0000", 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2490000000000004e-06 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8282 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149391+0000", 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990704145, 2024-09-13T05:02:28.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019880120000000001, 2024-09-13T05:02:28.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.152 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8282 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149391+0000", 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149391+0000", 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149390+0000", 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149392+0000", 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9300000000000002e-06 2024-09-13T05:02:28.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149398+0000", 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.118e-06 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149402+0000", 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1150000000000004e-06 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169259+0000", 2024-09-13T05:02:28.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019857377999999998 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169271+0000", 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1909999999999999e-05 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8283 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149405+0000", 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990689728, 2024-09-13T05:02:28.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019900592000000002, 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8283 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149405+0000", 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149405+0000", 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149405+0000", 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149410+0000", 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5999999999999997e-06 2024-09-13T05:02:28.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149416+0000", 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6189999999999999e-06 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149420+0000", 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.211e-06 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169297+0000", 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019876395000000002 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169306+0000", 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0969999999999995e-06 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8286 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149546+0000", 2024-09-13T05:02:28.160 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990548491, 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019860793000000002, 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8286 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149546+0000", 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149546+0000", 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149546+0000", 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149548+0000", 2024-09-13T05:02:28.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9989999999999998e-06 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149555+0000", 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9859999999999997e-06 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149560+0000", 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4859999999999999e-06 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169397+0000", 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019837621 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169407+0000", 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0026e-05 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.164 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8287 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149563+0000", 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990531628999999, 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019875645000000001, 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.165 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8287 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149563+0000", 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149563+0000", 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149563+0000", 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149565+0000", 2024-09-13T05:02:28.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1009999999999999e-06 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149572+0000", 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0149999999999998e-06 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149577+0000", 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.634e-06 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169432+0000", 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019854937999999999 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169439+0000", 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3139999999999998e-06 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.169 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8288 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149581+0000", 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990514277999999, 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019893628999999999, 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8288 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.170 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149581+0000", 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149581+0000", 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149580+0000", 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149582+0000", 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1220000000000002e-06 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149596+0000", 2024-09-13T05:02:28.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3542e-05 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149601+0000", 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9880000000000004e-06 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169464+0000", 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019863412 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169474+0000", 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9180000000000006e-06 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8289 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149605+0000", 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.99049005, 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019901083999999999, 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8289 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149605+0000", 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149605+0000", 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149605+0000", 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149607+0000", 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.12e-06 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149612+0000", 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5910000000000003e-06 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149617+0000", 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2989999999999998e-06 2024-09-13T05:02:28.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169498+0000", 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019881946000000001 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169506+0000", 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4850000000000003e-06 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8290 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149623+0000", 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990471841, 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019919554999999999, 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8290 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.179 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149623+0000", 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149623+0000", 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149623+0000", 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149625+0000", 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.018e-06 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149630+0000", 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5539999999999997e-06 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149635+0000", 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1880000000000004e-06 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169533+0000", 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019898049000000001 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169543+0000", 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0104e-05 2024-09-13T05:02:28.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8291 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149638+0000", 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990456697000001, 2024-09-13T05:02:28.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019938969000000001, 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8291 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149638+0000", 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149638+0000", 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149638+0000", 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149644+0000", 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6420000000000004e-06 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149649+0000", 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5280000000000003e-06 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149653+0000", 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3660000000000002e-06 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169568+0000", 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019914413999999998 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169577+0000", 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3710000000000006e-06 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8292 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149657+0000", 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990437548999999, 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019951170000000001, 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8292 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149657+0000", 2024-09-13T05:02:28.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149657+0000", 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149657+0000", 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149659+0000", 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.086e-06 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149666+0000", 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8079999999999999e-06 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149670+0000", 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4150000000000001e-06 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169601+0000", 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019931076999999998 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169609+0000", 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2409999999999998e-06 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.191 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8293 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149686+0000", 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990408913, 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019954685, 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8293 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149686+0000", 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149686+0000", 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149686+0000", 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149688+0000", 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3070000000000001e-06 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149694+0000", 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.302e-06 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149699+0000", 2024-09-13T05:02:28.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6500000000000004e-06 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169632+0000", 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019932722 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169641+0000", 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1300000000000007e-06 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8294 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149703+0000", 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990391741, 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968181000000002, 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8294 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149703+0000", 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149703+0000", 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149703+0000", 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149705+0000", 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2630000000000002e-06 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149710+0000", 2024-09-13T05:02:28.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4739999999999996e-06 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149715+0000", 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3880000000000002e-06 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169664+0000", 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019948857 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169671+0000", 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6620000000000005e-06 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8295 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149719+0000", 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990376232999999, 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019986182000000002, 2024-09-13T05:02:28.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8295 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149719+0000", 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149719+0000", 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149718+0000", 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149726+0000", 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8469999999999996e-06 2024-09-13T05:02:28.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149735+0000", 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7520000000000002e-06 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149741+0000", 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0499999999999997e-06 2024-09-13T05:02:28.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169697+0000", 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019956061000000001 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169705+0000", 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7980000000000006e-06 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8296 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149745+0000", 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990349702, 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01999074, 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8296 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149745+0000", 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149745+0000", 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149745+0000", 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149747+0000", 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1440000000000001e-06 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149753+0000", 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.468e-06 2024-09-13T05:02:28.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149760+0000", 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5209999999999997e-06 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169727+0000", 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967267 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169736+0000", 2024-09-13T05:02:28.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6810000000000004e-06 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8297 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149764+0000", 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990330891999999, 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020007239999999999, 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.209 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8297 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149764+0000", 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149764+0000", 2024-09-13T05:02:28.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149764+0000", 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149766+0000", 2024-09-13T05:02:28.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0849999999999999e-06 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149771+0000", 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6169999999999998e-06 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149776+0000", 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.194e-06 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169762+0000", 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019986090000000001 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169771+0000", 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6020000000000006e-06 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8298 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149807+0000", 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990287833, 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019999348, 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8298 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149807+0000", 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149807+0000", 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149807+0000", 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149809+0000", 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4449999999999999e-06 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149816+0000", 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6409999999999996e-06 2024-09-13T05:02:28.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149824+0000", 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3450000000000006e-06 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169799+0000", 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019974632999999999 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169806+0000", 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6669999999999996e-06 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8299 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149891+0000", 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990204387, 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019946116, 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.218 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8299 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149891+0000", 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149891+0000", 2024-09-13T05:02:28.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149890+0000", 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149893+0000", 2024-09-13T05:02:28.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.303e-06 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149902+0000", 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3799999999999994e-06 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149907+0000", 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.49e-06 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169830+0000", 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019922407 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169837+0000", 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1459999999999999e-06 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8300 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149912+0000", 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990183215, 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019960532999999999, 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8300 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149912+0000", 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149912+0000", 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149911+0000", 2024-09-13T05:02:28.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149913+0000", 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.108e-06 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149919+0000", 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5879999999999997e-06 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149923+0000", 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2359999999999998e-06 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169861+0000", 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019938083999999998 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169872+0000", 2024-09-13T05:02:28.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0916e-05 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:8301 3.1b 3.695ab93b (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e27)", 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.149927+0000", 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.990167848, 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019980876000000002, 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 8301 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149927+0000", 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149927+0000", 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149927+0000", 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149929+0000", 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9769999999999999e-06 2024-09-13T05:02:28.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149934+0000", 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4090000000000002e-06 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.149938+0000", 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0910000000000003e-06 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169899+0000", 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019960371000000001 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.169908+0000", 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4029999999999996e-06 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.231 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.232 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.232 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:28.233 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:28.234 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:28.233+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:28.334 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:28.334+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:28.380 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:28.381 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:28.389 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:28.435 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:28.435+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:28.536 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:28.536+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:28.536 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:28.536 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:28.537 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:28.537 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:28.545 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:28.577 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:28.638 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:28.636+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:28.653 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32275 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87241344899999995, 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010461302, 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32275 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:28.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780089+0000", 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2010000000000002e-06 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780095+0000", 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8019999999999997e-06 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780099+0000", 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3229999999999999e-06 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790415+0000", 2024-09-13T05:02:28.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010315649 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790475+0000", 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9843999999999998e-05 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790549+0000", 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3883999999999999e-05 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32276 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87239767999999995, 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010619966, 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32276 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:28.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780105+0000", 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.159e-06 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780111+0000", 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5639999999999996e-06 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780118+0000", 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790585+0000", 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010466197 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790648+0000", 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3927e-05 2024-09-13T05:02:28.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790723+0000", 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4635999999999995e-05 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32277 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8454144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:28.663 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87237870900000003, 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010784396999999999, 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32277 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780124+0000", 2024-09-13T05:02:28.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119e-06 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780129+0000", 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5799999999999999e-06 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780134+0000", 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4750000000000004e-06 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790750+0000", 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010616082000000001 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790824+0000", 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4031999999999999e-05 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790907+0000", 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2458000000000002e-05 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32278 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8683520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87233573499999995, 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010917582, 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32278 2024-09-13T05:02:28.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780164+0000", 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780169+0000", 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8300000000000003e-06 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780179+0000", 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0164e-05 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780186+0000", 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4799999999999998e-06 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790933+0000", 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010747358 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791001+0000", 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7390000000000004e-05 2024-09-13T05:02:28.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791083+0000", 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2177999999999998e-05 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32279 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8912896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87230968099999995, 2024-09-13T05:02:28.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011066273999999999, 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32279 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780193+0000", 2024-09-13T05:02:28.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780200+0000", 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3709999999999996e-06 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780205+0000", 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-06 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791108+0000", 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010903375 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791175+0000", 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6451999999999993e-05 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791257+0000", 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2645999999999994e-05 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32316 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8716288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780299+0000", 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87220199600000003, 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010340502, 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32316 2024-09-13T05:02:28.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780299+0000", 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780299+0000", 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780298+0000", 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780308+0000", 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1819999999999995e-06 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780314+0000", 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5710000000000003e-06 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780319+0000", 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6299999999999997e-06 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.789687+0000", 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0093680269999999993 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.789755+0000", 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8365e-05 2024-09-13T05:02:28.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790639+0000", 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088407899999999996 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32317 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8945664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87215346699999996, 2024-09-13T05:02:28.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010478401, 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32317 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780350+0000", 2024-09-13T05:02:28.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9790000000000002e-06 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780357+0000", 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2350000000000002e-06 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780362+0000", 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790666+0000", 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010303911000000001 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790735+0000", 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8396e-05 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790826+0000", 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0983999999999995e-05 2024-09-13T05:02:28.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32318 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9175040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87213374600000004, 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010623140999999999, 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32318 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780369+0000", 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5620000000000002e-06 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780375+0000", 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780380+0000", 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6879999999999998e-06 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790855+0000", 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010474957 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790920+0000", 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5030999999999995e-05 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790990+0000", 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0384999999999998e-05 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.690 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32337 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87210406699999998, 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011667398000000001, 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32337 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780396+0000", 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780399+0000", 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9849999999999998e-06 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780406+0000", 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780415+0000", 2024-09-13T05:02:28.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6799999999999999e-06 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791286+0000", 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010870869999999999 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791955+0000", 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066933100000000001 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792064+0000", 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010907499999999999 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32338 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7798784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87208100700000002, 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011830083, 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.695 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32338 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:28.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780419+0000", 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780422+0000", 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.339e-06 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780428+0000", 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0730000000000001e-06 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780433+0000", 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7010000000000004e-06 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792095+0000", 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011662529 2024-09-13T05:02:28.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792164+0000", 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9346999999999999e-05 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792250+0000", 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5504e-05 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32339 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8028160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87206403799999999, 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011996256, 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.700 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32339 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780436+0000", 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780439+0000", 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.198e-06 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780444+0000", 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.806e-06 2024-09-13T05:02:28.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780451+0000", 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792275+0000", 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011824573999999999 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792339+0000", 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3628999999999997e-05 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792433+0000", 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4029000000000004e-05 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32340 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8257536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.704 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87204598499999997, 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012155473999999999, 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32340 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:28.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780457+0000", 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780462+0000", 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7490000000000004e-06 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780467+0000", 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5970000000000002e-06 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792461+0000", 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011993668000000001 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792529+0000", 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8249000000000003e-05 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792610+0000", 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1451000000000003e-05 2024-09-13T05:02:28.708 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32341 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8486912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87202963099999997, 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012301632999999999, 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32341 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780473+0000", 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3269999999999999e-06 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780481+0000", 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7970000000000001e-06 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780485+0000", 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.408e-06 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792636+0000", 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012150879 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792699+0000", 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2842000000000006e-05 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792773+0000", 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000001e-05 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32342 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8716288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:28.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87201071799999996, 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012452926, 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32342 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:28.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780493+0000", 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9979999999999999e-06 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780499+0000", 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-06 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780504+0000", 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6909999999999996e-06 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792805+0000", 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012301288 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792868+0000", 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3202999999999998e-05 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792943+0000", 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4592999999999995e-05 2024-09-13T05:02:28.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32343 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8945664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87199186699999998, 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012601064, 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32343 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780511+0000", 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.215e-06 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780516+0000", 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6380000000000001e-06 2024-09-13T05:02:28.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780521+0000", 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792968+0000", 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012446262 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793031+0000", 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3609999999999996e-05 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793110+0000", 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8910000000000002e-05 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32344 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 9175040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:28.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.871975316, 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012769502, 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32344 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780525+0000", 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780527+0000", 2024-09-13T05:02:28.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3649999999999998e-06 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780534+0000", 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5980000000000002e-06 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780538+0000", 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.194e-06 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793139+0000", 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012600731 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793210+0000", 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1131999999999996e-05 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793295+0000", 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4938000000000006e-05 2024-09-13T05:02:28.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32379 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8749056~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87093520000000002, 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010495398, 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.727 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32379 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781565+0000", 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781568+0000", 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781574+0000", 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2929999999999997e-06 2024-09-13T05:02:28.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781579+0000", 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0340000000000004e-06 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791907+0000", 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010328027 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791977+0000", 2024-09-13T05:02:28.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9728999999999996e-05 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792061+0000", 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4332000000000005e-05 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32380 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8978432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87091738500000004, 2024-09-13T05:02:28.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010658737, 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32380 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781585+0000", 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0279999999999999e-06 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781589+0000", 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781591+0000", 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6120000000000001e-06 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792087+0000", 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010496152 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792161+0000", 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3473999999999995e-05 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792242+0000", 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1403999999999995e-05 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.735 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32381 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9207808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87090752900000001, 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012026156, 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.736 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32381 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781594+0000", 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3850000000000001e-06 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781597+0000", 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9579999999999999e-06 2024-09-13T05:02:28.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781600+0000", 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4169999999999999e-06 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792270+0000", 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010670059000000001 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792339+0000", 2024-09-13T05:02:28.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8861000000000004e-05 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793619+0000", 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001280712 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32382 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9437184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:28.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.87089849799999997, 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012273719000000001, 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32382 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:28.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781604+0000", 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.905e-06 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781610+0000", 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3799999999999999e-06 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781614+0000", 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5870000000000001e-06 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793687+0000", 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012073084 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793763+0000", 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6074000000000004e-05 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793876+0000", 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011301199999999999 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.744 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:28.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:28.745 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:28.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:28.747 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:28.747 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:28.747 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:28.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:28.748 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:28.748 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:28.746+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:28.759 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:28.760 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:28.776 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:28.776 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:28.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:28.793 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:28.810 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:28.811 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:28.827 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:28.828 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:28.847 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:28.847+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:28.867 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:28.867 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:28.876 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:28.947 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:28.947+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:28.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:02:28.998 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:28.999 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:28.999 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T05:02:28.999 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:02:28.999 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:02:28.999 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T05:02:29.005 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone12 from 0xc2a156b (overlap 0x3c9e487) to 0x122f7c2 2024-09-13T05:02:29.049 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:29.048+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:29.056 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.057 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.068 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:29.138 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:29.139 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "" 2024-09-13T05:02:29.139 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:29.164 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-13T05:02:29.189 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:29.188+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:29.205 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.205 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:29.250 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:29.250+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:29.294 INFO:teuthology.orchestra.run.smithi053.stdout:1543 punch from 0x3ff1ce5 to 0x3ff3015, (0x1330 bytes) 2024-09-13T05:02:29.297 INFO:teuthology.orchestra.run.smithi053.stdout:1544 read 0x1a274cd thru 0x1a33f60 (0xca94 bytes) 2024-09-13T05:02:29.298 INFO:teuthology.orchestra.run.smithi053.stdout:1545 read 0x73b0f20 thru 0x73b560e (0x46ef bytes) 2024-09-13T05:02:29.298 INFO:teuthology.orchestra.run.smithi053.stdout:1546 writesame 0xaf132ae thru 0xaf20b07 data_size 0x2(0xd85a bytes) 2024-09-13T05:02:29.314 INFO:teuthology.orchestra.run.smithi053.stdout:1547 read 0x65589a4 thru 0x655c38c (0x39e9 bytes) 2024-09-13T05:02:29.315 INFO:teuthology.orchestra.run.smithi053.stdout:1548 write 0x4f98832 thru 0x4f9ac88 (0x2457 bytes) 2024-09-13T05:02:29.321 INFO:teuthology.orchestra.run.smithi053.stdout:1549 write 0x8a14c47 thru 0x8a1b5f8 (0x69b2 bytes) 2024-09-13T05:02:29.330 INFO:teuthology.orchestra.run.smithi053.stdout:1551 write 0x2eab98f thru 0x2eb1f61 (0x65d3 bytes) 2024-09-13T05:02:29.339 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.339 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:29.350 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:29.350+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:29.369 INFO:teuthology.orchestra.run.smithi053.stdout:1552 punch from 0x1d1afc9 to 0x1d27247, (0xc27e bytes) 2024-09-13T05:02:29.369 INFO:teuthology.orchestra.run.smithi053.stdout:1553 write 0x1ce1cd1 thru 0x1cebffc (0xa32c bytes) 2024-09-13T05:02:29.426 INFO:teuthology.orchestra.run.smithi053.stdout:1555 read 0x79792a5 thru 0x797b8fa (0x2656 bytes) 2024-09-13T05:02:29.427 INFO:teuthology.orchestra.run.smithi053.stdout:1556 read 0x5fa606c thru 0x5fae8f5 (0x888a bytes) 2024-09-13T05:02:29.427 INFO:teuthology.orchestra.run.smithi053.stdout:1557 punch from 0x3db34af to 0x3dbd339, (0x9e8a bytes) 2024-09-13T05:02:29.427 INFO:teuthology.orchestra.run.smithi053.stdout:1558 write 0x8d44a75 thru 0x8d4f066 (0xa5f2 bytes) 2024-09-13T05:02:29.435 INFO:teuthology.orchestra.run.smithi053.stdout:1559 clone 14 order 18 su 8192 sc 14 2024-09-13T05:02:29.452 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:29.451+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:29.473 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.473 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.481 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:29.552 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:29.552+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:29.629 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.629 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:29.653 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:29.653+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:29.740 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.740 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.749 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:29.793 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:29.792+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:29.850 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:29.850 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:29.856 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:29.855+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:29.859 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:29.949 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:02:29.956 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:29.956+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:30.006 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.006 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.012 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone13 from 0xc2a156b (overlap 0xc2a156b) to 0x3a639a0 2024-09-13T05:02:30.016 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:30.058 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:30.057+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:30.158 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:30.158+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:30.159 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-13T05:02:30.163 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.163 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.173 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:30.190 INFO:teuthology.orchestra.run.smithi053.stdout:1561 write 0x8351b3e thru 0x8359e54 (0x8317 bytes) 2024-09-13T05:02:30.222 INFO:teuthology.orchestra.run.smithi053.stdout:1562 writesame 0xe48e2c1 thru 0xe496141 data_size 0x55(0x7e81 bytes) 2024-09-13T05:02:30.222 INFO:teuthology.orchestra.run.smithi053.stdout:1563 write 0xffb224 thru 0xffd619 (0x23f6 bytes) 2024-09-13T05:02:30.233 INFO:teuthology.orchestra.run.smithi053.stdout:1564 write 0x1705f79 thru 0x1708494 (0x251c bytes) 2024-09-13T05:02:30.245 INFO:teuthology.orchestra.run.smithi053.stdout:1565 write 0xd131b4a thru 0xd134de5 (0x329c bytes) 2024-09-13T05:02:30.259 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:30.259+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:30.263 INFO:teuthology.orchestra.run.smithi053.stdout:1566 trunc from 0xe496142 to 0x1dbd49a 2024-09-13T05:02:30.291 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.292 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.301 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:30.322 INFO:teuthology.orchestra.run.smithi053.stdout:1567 read 0x1a2fdc3 thru 0x1a34353 (0x4591 bytes) 2024-09-13T05:02:30.324 INFO:teuthology.orchestra.run.smithi053.stdout:1568 read 0x14d8d9e thru 0x14e49e0 (0xbc43 bytes) 2024-09-13T05:02:30.331 INFO:teuthology.orchestra.run.smithi053.stdout:1571 write 0x24c138c thru 0x24ca236 (0x8eab bytes) 2024-09-13T05:02:30.337 INFO:teuthology.orchestra.run.smithi053.stdout:1573 trunc from 0x24ca237 to 0x1acb2aa 2024-09-13T05:02:30.360 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:30.359+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:30.390 INFO:teuthology.orchestra.run.smithi053.stdout:1575 write 0x1793872 thru 0x1795ee3 (0x2672 bytes) 2024-09-13T05:02:30.400 INFO:teuthology.orchestra.run.smithi053.stdout:1576 trunc from 0x1acb2aa to 0x94226b3 2024-09-13T05:02:30.405 INFO:teuthology.orchestra.run.smithi053.stdout:1577 write 0x203b97c thru 0x204488d (0x8f12 bytes) 2024-09-13T05:02:30.412 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.412 INFO:teuthology.orchestra.run.smithi053.stdout:1580 writesame 0x6b4226d thru 0x6b4a728 data_size 0xa(0x84bc bytes) 2024-09-13T05:02:30.417 INFO:teuthology.orchestra.run.smithi053.stdout:1581 writesame 0x6056cfe thru 0x6066c47 data_size 0x29(0xff4a bytes) 2024-09-13T05:02:30.418 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.418 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.428 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:30.435 INFO:teuthology.orchestra.run.smithi053.stdout:1582 write 0xaeca6ef thru 0xaece99d (0x42af bytes) 2024-09-13T05:02:30.439 INFO:teuthology.orchestra.run.smithi053.stdout:1583 write 0xac85ccd thru 0xac8b552 (0x5886 bytes) 2024-09-13T05:02:30.444 INFO:teuthology.orchestra.run.smithi053.stdout:1584 write 0x596b131 thru 0x5979f32 (0xee02 bytes) 2024-09-13T05:02:30.452 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.460 INFO:teuthology.orchestra.run.smithi053.stdout:1591 write 0xc6d23d3 thru 0xc6e05d2 (0xe200 bytes) 2024-09-13T05:02:30.460 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:30.460+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:30.469 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1593 punch from 0xdcf3c to 0xe714d, (0xa211 bytes) 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1594 punch from 0x71b47ca to 0x71bc7f1, (0x8027 bytes) 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1595 punch from 0x3431494 to 0x34348d4, (0x3440 bytes) 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1596 read 0xc328762 thru 0xc328fd5 (0x874 bytes) 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1597 read 0x65823ad thru 0x658b97d (0x95d1 bytes) 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1600 read 0x3b9c147 thru 0x3baa2d7 (0xe191 bytes) 2024-09-13T05:02:30.470 INFO:teuthology.orchestra.run.smithi053.stdout:1601 write 0x6f51b43 thru 0x6f52409 (0x8c7 bytes) 2024-09-13T05:02:30.477 INFO:teuthology.orchestra.run.smithi053.stdout:1602 trunc from 0xc6e05d3 to 0x85f976f 2024-09-13T05:02:30.488 INFO:teuthology.orchestra.run.smithi053.stdout:1603 writesame 0x2a3328 thru 0x2b1bd1 data_size 0x1b(0xe8aa bytes) 2024-09-13T05:02:30.519 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.526 INFO:teuthology.orchestra.run.smithi053.stdout:1605 trunc from 0x85f976f to 0xe3ad990 2024-09-13T05:02:30.531 INFO:teuthology.orchestra.run.smithi053.stdout:1607 read 0x5df7de4 thru 0x5dff546 (0x7763 bytes) 2024-09-13T05:02:30.531 INFO:teuthology.orchestra.run.smithi053.stdout:1611 read 0x283b952 thru 0x28457da (0x9e89 bytes) 2024-09-13T05:02:30.539 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.539 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.546 INFO:teuthology.orchestra.run.smithi053.stdout:1613 trunc from 0xe3ad990 to 0xb134bc 2024-09-13T05:02:30.548 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:30.562 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:30.561+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:30.605 INFO:teuthology.orchestra.run.smithi053.stdout:1614 read 0xab917c thru 0xabb905 (0x278a bytes) 2024-09-13T05:02:30.605 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.618 INFO:teuthology.orchestra.run.smithi053.stdout:1617 write 0x819a874 thru 0x819f8cd (0x505a bytes) 2024-09-13T05:02:30.622 INFO:teuthology.orchestra.run.smithi053.stdout:1618 punch from 0x3de1aa1 to 0x3def65d, (0xdbbc bytes) 2024-09-13T05:02:30.623 INFO:teuthology.orchestra.run.smithi053.stdout:1619 write 0x41270b4 thru 0x4130ff1 (0x9f3e bytes) 2024-09-13T05:02:30.630 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.630 INFO:teuthology.orchestra.run.smithi053.stdout:1621 write 0x138b419 thru 0x13906bf (0x52a7 bytes) 2024-09-13T05:02:30.643 INFO:teuthology.orchestra.run.smithi053.stdout:1622 write 0xa59ee26 thru 0xa5aa35a (0xb535 bytes) 2024-09-13T05:02:30.657 INFO:teuthology.orchestra.run.smithi053.stdout:1623 writesame 0xb9c5ae0 thru 0xb9ca5d8 data_size 0x1(0x4af9 bytes) 2024-09-13T05:02:30.657 INFO:teuthology.orchestra.run.smithi053.stdout:1624 write 0x4484b82 thru 0x448fdfe (0xb27d bytes) 2024-09-13T05:02:30.663 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:30.663+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:30.664 INFO:teuthology.orchestra.run.smithi053.stdout:1626 read 0x5a7e744 thru 0x5a834f1 (0x4dae bytes) 2024-09-13T05:02:30.664 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.664 INFO:teuthology.orchestra.run.smithi053.stdout:1629 read 0x4e62443 thru 0x4e6ca21 (0xa5df bytes) 2024-09-13T05:02:30.664 INFO:teuthology.orchestra.run.smithi053.stdout:1631 write 0x49be3e5 thru 0x49be6a5 (0x2c1 bytes) 2024-09-13T05:02:30.666 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.667 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.676 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:30.678 INFO:teuthology.orchestra.run.smithi053.stdout:1632 trunc from 0xb9ca5d9 to 0x1a69d92 2024-09-13T05:02:30.701 INFO:teuthology.orchestra.run.smithi053.stdout:1633 writesame 0x510950c thru 0x510aa98 data_size 0x3(0x158d bytes) 2024-09-13T05:02:30.702 INFO:teuthology.orchestra.run.smithi053.stdout:1634 trunc from 0x510aa99 to 0x4863647 2024-09-13T05:02:30.707 INFO:teuthology.orchestra.run.smithi053.stdout:1635 write 0xc06990 thru 0xc10854 (0x9ec5 bytes) 2024-09-13T05:02:30.712 INFO:teuthology.orchestra.run.smithi053.stdout:1636 trunc from 0x4863647 to 0x26e1c0c 2024-09-13T05:02:30.718 INFO:teuthology.orchestra.run.smithi053.stdout:1638 trunc from 0x26e1c0c to 0x3db804e 2024-09-13T05:02:30.724 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.724 INFO:teuthology.orchestra.run.smithi053.stdout:1643 write 0x2ab6e6e thru 0x2abbc18 (0x4dab bytes) 2024-09-13T05:02:30.731 INFO:teuthology.orchestra.run.smithi053.stdout:1644 trunc from 0x3db804e to 0x74e4ae4 2024-09-13T05:02:30.736 INFO:teuthology.orchestra.run.smithi053.stdout:1646 read 0x2e20cf6 thru 0x2e2e90e (0xdc19 bytes) 2024-09-13T05:02:30.743 INFO:teuthology.orchestra.run.smithi053.stdout:1647 trunc from 0x74e4ae4 to 0xd149a82 2024-09-13T05:02:30.748 INFO:teuthology.orchestra.run.smithi053.stdout:1648 read 0x4be9954 thru 0x4bf81e5 (0xe892 bytes) 2024-09-13T05:02:30.748 INFO:teuthology.orchestra.run.smithi053.stdout:1649 write 0x9cb7b13 thru 0x9cc5a0e (0xdefc bytes) 2024-09-13T05:02:30.757 INFO:teuthology.orchestra.run.smithi053.stdout:1650 writesame 0x74d095f thru 0x74db923 data_size 0x35(0xafc5 bytes) 2024-09-13T05:02:30.764 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:30.763+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:30.774 INFO:teuthology.orchestra.run.smithi053.stdout:1651 trunc from 0xd149a82 to 0x340c6e0 2024-09-13T05:02:30.786 INFO:teuthology.orchestra.run.smithi053.stdout:1652 punch from 0x16e20ad to 0x16eb941, (0x9894 bytes) 2024-09-13T05:02:30.788 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.788 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.797 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:30.804 INFO:teuthology.orchestra.run.smithi053.stdout:1653 writesame 0xd91eed8 thru 0xd92dbb8 data_size 0x1(0xece1 bytes) 2024-09-13T05:02:30.804 INFO:teuthology.orchestra.run.smithi053.stdout:1655 read 0xa1ae54b thru 0xa1bc666 (0xe11c bytes) 2024-09-13T05:02:30.804 INFO:teuthology.orchestra.run.smithi053.stdout:1657 trunc from 0xd92dbb9 to 0xd70fad0 2024-09-13T05:02:30.809 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.809 INFO:teuthology.orchestra.run.smithi053.stdout:1660 write 0x5eacb09 thru 0x5ebc6a6 (0xfb9e bytes) 2024-09-13T05:02:30.822 INFO:teuthology.orchestra.run.smithi053.stdout:1661 trunc from 0xd70fad0 to 0xa708c22 2024-09-13T05:02:30.827 INFO:teuthology.orchestra.run.smithi053.stdout:1663 writesame 0xa36b3ca thru 0xa3729a4 data_size 0x59(0x75db bytes) 2024-09-13T05:02:30.839 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:30.844 INFO:teuthology.orchestra.run.smithi053.stdout:1664 trunc from 0xa708c22 to 0x19bccc4 2024-09-13T05:02:30.866 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:30.865+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:30.874 INFO:teuthology.orchestra.run.smithi053.stdout:1665 writesame 0xb5243a0 thru 0xb52d9f1 data_size 0x47(0x9652 bytes) 2024-09-13T05:02:30.874 INFO:teuthology.orchestra.run.smithi053.stdout:1666 writesame 0x6998c72 thru 0x699ca89 data_size 0x4(0x3e18 bytes) 2024-09-13T05:02:30.879 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.879 INFO:teuthology.orchestra.run.smithi053.stdout:1670 read 0x237881a thru 0x2384c95 (0xc47c bytes) 2024-09-13T05:02:30.883 INFO:teuthology.orchestra.run.smithi053.stdout:1671 trunc from 0xb52d9f2 to 0x75158c8 2024-09-13T05:02:30.896 INFO:teuthology.orchestra.run.smithi053.stdout:1672 trunc from 0x75158c8 to 0x11dd03f 2024-09-13T05:02:30.907 INFO:teuthology.orchestra.run.smithi053.stdout:1674 punch from 0x110cc56 to 0x111c866, (0xfc10 bytes) 2024-09-13T05:02:30.920 INFO:teuthology.orchestra.run.smithi053.stdout:1675 write 0x83a55ae thru 0x83b1880 (0xc2d3 bytes) 2024-09-13T05:02:30.928 INFO:teuthology.orchestra.run.smithi053.stdout:1676 read 0x6a8d666 thru 0x6a9c516 (0xeeb1 bytes) 2024-09-13T05:02:30.928 INFO:teuthology.orchestra.run.smithi053.stdout:1678 read 0xe48b9a thru 0xe517ac (0x8c13 bytes) 2024-09-13T05:02:30.929 INFO:teuthology.orchestra.run.smithi053.stdout:1680 write 0x6198b12 thru 0x619abad (0x209c bytes) 2024-09-13T05:02:30.944 INFO:teuthology.orchestra.run.smithi053.stdout:1681 write 0xeb34b7e thru 0xeb38c3a (0x40bd bytes) 2024-09-13T05:02:30.949 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:30.949 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:30.949 INFO:teuthology.orchestra.run.smithi053.stdout:1684 write 0xac344d9 thru 0xac3b662 (0x718a bytes) 2024-09-13T05:02:30.954 INFO:teuthology.orchestra.run.smithi053.stdout:1686 write 0x75494fe thru 0x7558fd7 (0xfada bytes) 2024-09-13T05:02:30.960 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:30.965 INFO:teuthology.orchestra.run.smithi053.stdout:1687 read 0x73b8891 thru 0x73bd88b (0x4ffb bytes) 2024-09-13T05:02:30.966 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:30.971 INFO:teuthology.orchestra.run.smithi053.stdout:1691 trunc from 0xeb38c3b to 0x9893ccf 2024-09-13T05:02:30.981 INFO:teuthology.orchestra.run.smithi053.stdout:1692 writesame 0x601a373 thru 0x6028dd5 data_size 0x9(0xea63 bytes) 2024-09-13T05:02:30.992 INFO:teuthology.orchestra.run.smithi053.stdout:1694 write 0x5443ab4 thru 0x5451ac7 (0xe014 bytes) 2024-09-13T05:02:30.993 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:30.993 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:31.002 INFO:teuthology.orchestra.run.smithi053.stdout:1695 punch from 0x3e74260 to 0x3e77a51, (0x37f1 bytes) 2024-09-13T05:02:31.002 INFO:teuthology.orchestra.run.smithi053.stdout:1696 read 0x41df875 thru 0x41e9c83 (0xa40f bytes) 2024-09-13T05:02:31.002 INFO:teuthology.orchestra.run.smithi053.stdout:1698 write 0x13ed84e thru 0x13efffc (0x27af bytes) 2024-09-13T05:02:31.004 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:31.004+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:31.011 INFO:teuthology.orchestra.run.smithi053.stdout:1702 trunc from 0x9893ccf to 0xed5a469 2024-09-13T05:02:31.015 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:31.015 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:31.016 INFO:teuthology.orchestra.run.smithi053.stdout:1704 punch from 0xd7c711d to 0xd7d1522, (0xa405 bytes) 2024-09-13T05:02:31.021 INFO:teuthology.orchestra.run.smithi053.stdout:1705 trunc from 0xed5a469 to 0xafd8919 2024-09-13T05:02:31.028 INFO:teuthology.orchestra.run.smithi053.stdout:1706 trunc from 0xafd8919 to 0xc4f8e14 2024-09-13T05:02:31.032 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:31.032 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:31.033 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.036 INFO:teuthology.orchestra.run.smithi053.stdout:1710 trunc from 0xc4f8e14 to 0x9e761f3 2024-09-13T05:02:31.041 INFO:teuthology.orchestra.run.smithi053.stdout:1711 read 0x23148db thru 0x231a030 (0x5756 bytes) 2024-09-13T05:02:31.044 INFO:teuthology.orchestra.run.smithi053.stdout:1712 trunc from 0x9e761f3 to 0xc9edc7e 2024-09-13T05:02:31.049 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:31.050 INFO:teuthology.orchestra.run.smithi053.stdout:1713 write 0xc8f6ce4 thru 0xc8f8889 (0x1ba6 bytes) 2024-09-13T05:02:31.050 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:31.053 INFO:teuthology.orchestra.run.smithi053.stdout:1714 read 0x3b7460e thru 0x3b83d2a (0xf71d bytes) 2024-09-13T05:02:31.053 INFO:teuthology.orchestra.run.smithi053.stdout:1715 writesame 0x706256f thru 0x7068cf6 data_size 0x4(0x6788 bytes) 2024-09-13T05:02:31.059 INFO:teuthology.orchestra.run.smithi053.stdout:1716 writesame 0x8fcaab2 thru 0x8fd71b1 data_size 0x80(0xc700 bytes) 2024-09-13T05:02:31.066 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:31.066+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:31.068 INFO:teuthology.orchestra.run.smithi053.stdout:1717 punch from 0x8710576 to 0x87167a9, (0x6233 bytes) 2024-09-13T05:02:31.068 INFO:teuthology.orchestra.run.smithi053.stdout:1718 write 0x7614c0a thru 0x761b710 (0x6b07 bytes) 2024-09-13T05:02:31.070 INFO:teuthology.orchestra.run.smithi053.stdout:1720 writesame 0x280baa5 thru 0x2819e98 data_size 0x12(0xe3f4 bytes) 2024-09-13T05:02:31.071 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:31.071 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:31.077 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:31.077 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:31.079 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.079 INFO:teuthology.orchestra.run.smithi053.stdout:1722 read 0x28a0abc thru 0x28aba98 (0xafdd bytes) 2024-09-13T05:02:31.080 INFO:teuthology.orchestra.run.smithi053.stdout:1723 writesame 0x8b662c thru 0x8c42fe data_size 0x1(0xdcd3 bytes) 2024-09-13T05:02:31.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:31.095 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:31.096 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:31.116 INFO:teuthology.orchestra.run.smithi053.stdout:1724 writesame 0x8b92a00 thru 0x8b9d475 data_size 0x15(0xaa76 bytes) 2024-09-13T05:02:31.118 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:31.119 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:31.127 INFO:teuthology.orchestra.run.smithi053.stdout:1726 trunc from 0xc9edc7e to 0x9888bd9 2024-09-13T05:02:31.138 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:31.138 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:31.146 INFO:teuthology.orchestra.run.smithi053.stdout:1728 trunc from 0x9888bd9 to 0x22474fb 2024-09-13T05:02:31.166 INFO:teuthology.orchestra.run.smithi053.stdout:1729 punch from 0x5f5c2e to 0x605417, (0xf7e9 bytes) 2024-09-13T05:02:31.172 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:31.172+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:31.179 INFO:teuthology.orchestra.run.smithi053.stdout:1730 write 0x88a71bc thru 0x88ae033 (0x6e78 bytes) 2024-09-13T05:02:31.185 INFO:teuthology.orchestra.run.smithi053.stdout:1731 writesame 0x3159e84 thru 0x3163fd1 data_size 0xb(0xa14e bytes) 2024-09-13T05:02:31.198 INFO:teuthology.orchestra.run.smithi053.stdout:1733 trunc from 0x88ae034 to 0xd6b6584 2024-09-13T05:02:31.204 INFO:teuthology.orchestra.run.smithi053.stdout:1736 read 0x761a7a5 thru 0x7628cfc (0xe558 bytes) 2024-09-13T05:02:31.204 INFO:teuthology.orchestra.run.smithi053.stdout:1738 read 0xd47a642 thru 0xd489b8b (0xf54a bytes) 2024-09-13T05:02:31.204 INFO:teuthology.orchestra.run.smithi053.stdout:1739 read 0xafc1469 thru 0xafc38a6 (0x243e bytes) 2024-09-13T05:02:31.204 INFO:teuthology.orchestra.run.smithi053.stdout:1742 read 0x5d8daed thru 0x5d8fb86 (0x209a bytes) 2024-09-13T05:02:31.204 INFO:teuthology.orchestra.run.smithi053.stdout:1743 punch from 0x81207e7 to 0x812294c, (0x2165 bytes) 2024-09-13T05:02:31.210 INFO:teuthology.orchestra.run.smithi053.stdout:1744 write 0xe9a5fdd thru 0xe9b029b (0xa2bf bytes) 2024-09-13T05:02:31.218 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.218 INFO:teuthology.orchestra.run.smithi053.stdout:1746 punch from 0x9090ab to 0x916ff2, (0xdf47 bytes) 2024-09-13T05:02:31.218 INFO:teuthology.orchestra.run.smithi053.stdout:1747 write 0xa51e8e4 thru 0xa526736 (0x7e53 bytes) 2024-09-13T05:02:31.223 INFO:teuthology.orchestra.run.smithi053.stdout:1748 read 0x603d2ba thru 0x60494dc (0xc223 bytes) 2024-09-13T05:02:31.223 INFO:teuthology.orchestra.run.smithi053.stdout:1749 read 0x7783815 thru 0x7791342 (0xdb2e bytes) 2024-09-13T05:02:31.224 INFO:teuthology.orchestra.run.smithi053.stdout:1750 write 0xbb6683 thru 0xbbec43 (0x85c1 bytes) 2024-09-13T05:02:31.226 INFO:teuthology.orchestra.run.smithi053.stdout:1752 read 0x9365668 thru 0x9371f93 (0xc92c bytes) 2024-09-13T05:02:31.226 INFO:teuthology.orchestra.run.smithi053.stdout:1754 read 0x42c77c3 thru 0x42cb150 (0x398e bytes) 2024-09-13T05:02:31.226 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.226 INFO:teuthology.orchestra.run.smithi053.stdout:1757 writesame 0xd73ec1 thru 0xd82793 data_size 0x1(0xe8d3 bytes) 2024-09-13T05:02:31.233 INFO:teuthology.orchestra.run.smithi053.stdout:1758 read 0x4160967 thru 0x416a51b (0x9bb5 bytes) 2024-09-13T05:02:31.235 INFO:teuthology.orchestra.run.smithi053.stdout:1759 trunc from 0xe9b029c to 0xcae1e33 2024-09-13T05:02:31.244 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.245 INFO:teuthology.orchestra.run.smithi053.stdout:1762 read 0x37855c2 thru 0x378efef (0x9a2e bytes) 2024-09-13T05:02:31.247 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0174563509999999, 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.248 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:31.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:02:31.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0165282419999997, 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.253 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0154298329999998, 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:02:31.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0093698719999997, 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.262 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:02:31.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:02:31.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:31.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0093608769999998, 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:02:31.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0093525059999999, 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:02:31.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0093424850000003, 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.277 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:31.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.281 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0093267619999997, 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:02:31.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0074245929999996, 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.286 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:31.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.290 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0074166340000001, 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:02:31.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.007408946, 2024-09-13T05:02:31.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:02:31.299 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0073982539999999, 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.300 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.304 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0073900130000002, 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:02:31.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.007366352, 2024-09-13T05:02:31.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:02:31.313 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0057238929999999, 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.314 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.318 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0057156599999999, 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:02:31.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:02:31.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.005492931, 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.327 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.005480382, 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:02:31.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:02:31.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0034411560000001, 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:02:31.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0034313770000001, 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:31.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:31.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:02:31.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:31.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:31.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:31.344 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:31.346 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:31.345+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:31.347 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1763 trunc from 0xcae1e33 to 0xa14dbf 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1765 punch from 0x430801 to 0x43b5c0, (0xadbf bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1766 read 0x47eaa5 thru 0x4809e5 (0x1f41 bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1767 read 0x90f3f9 thru 0x9133fa (0x4002 bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1769 trunc from 0xa14dbf to 0x8bfc5f1 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1770 read 0x3cebdd9 thru 0x3cf487e (0x8aa6 bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1771 read 0x4d57d01 thru 0x4d63b16 (0xbe16 bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1775 read 0x5b27119 thru 0x5b2aca9 (0x3b91 bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1776 writesame 0x961e338 thru 0x962ae7c data_size 0x1(0xcb45 bytes) 2024-09-13T05:02:31.348 INFO:teuthology.orchestra.run.smithi053.stdout:1777 trunc from 0x962ae7d to 0x58ad466 2024-09-13T05:02:31.349 INFO:teuthology.orchestra.run.smithi053.stdout:1778 trunc from 0x58ad466 to 0x3128066 2024-09-13T05:02:31.349 INFO:teuthology.orchestra.run.smithi053.stdout:1779 read 0x1be60b thru 0x1c27df (0x41d5 bytes) 2024-09-13T05:02:31.349 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.349 INFO:teuthology.orchestra.run.smithi053.stdout:1783 trunc from 0x3128066 to 0x9d0e27f 2024-09-13T05:02:31.357 INFO:teuthology.orchestra.run.smithi053.stdout:1784 write 0xd9d5ca7 thru 0xd9de11b (0x8475 bytes) 2024-09-13T05:02:31.364 INFO:teuthology.orchestra.run.smithi053.stdout:1787 writesame 0x3160d56 thru 0x31646fb data_size 0x2f(0x39a6 bytes) 2024-09-13T05:02:31.368 INFO:teuthology.orchestra.run.smithi053.stdout:1788 punch from 0x85d652d to 0x85de00e, (0x7ae1 bytes) 2024-09-13T05:02:31.368 INFO:teuthology.orchestra.run.smithi053.stdout:1789 writesame 0x1676ba1 thru 0x16851a9 data_size 0x1(0xe609 bytes) 2024-09-13T05:02:31.384 INFO:teuthology.orchestra.run.smithi053.stdout:1791 writesame 0xd316c47 thru 0xd321d90 data_size 0xb(0xb14a bytes) 2024-09-13T05:02:31.393 INFO:teuthology.orchestra.run.smithi053.stdout:1792 read 0xae8116 thru 0xaf3c2f (0xbb1a bytes) 2024-09-13T05:02:31.393 INFO:teuthology.orchestra.run.smithi053.stdout:1793 read 0xb7baa06 thru 0xb7baf5b (0x556 bytes) 2024-09-13T05:02:31.395 INFO:teuthology.orchestra.run.smithi053.stdout:1797 trunc from 0xd9de11c to 0xc308f3e 2024-09-13T05:02:31.405 INFO:teuthology.orchestra.run.smithi053.stdout:1798 read 0x645dc thru 0x67058 (0x2a7d bytes) 2024-09-13T05:02:31.406 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.406 INFO:teuthology.orchestra.run.smithi053.stdout:1801 writesame 0x12889f5 thru 0x1295bac data_size 0xc(0xd1b8 bytes) 2024-09-13T05:02:31.414 INFO:teuthology.orchestra.run.smithi053.stdout:1802 write 0x11d0b65 thru 0x11d3545 (0x29e1 bytes) 2024-09-13T05:02:31.417 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.417 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.417 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.417 INFO:teuthology.orchestra.run.smithi053.stdout:1806 punch from 0x11c75a2 to 0x11d1e8d, (0xa8eb bytes) 2024-09-13T05:02:31.423 INFO:teuthology.orchestra.run.smithi053.stdout:1809 write 0xce86465 thru 0xce86fb1 (0xb4d bytes) 2024-09-13T05:02:31.426 INFO:teuthology.orchestra.run.smithi053.stdout:1810 read 0x32afaef thru 0x32b3338 (0x384a bytes) 2024-09-13T05:02:31.434 INFO:teuthology.orchestra.run.smithi053.stdout:1811 trunc from 0xce86fb2 to 0x7525859 2024-09-13T05:02:31.446 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:31.446+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:31.459 INFO:teuthology.orchestra.run.smithi053.stdout:1813 write 0xe153707 thru 0xe161abd (0xe3b7 bytes) 2024-09-13T05:02:31.469 INFO:teuthology.orchestra.run.smithi053.stdout:1814 punch from 0x24c36b3 to 0x24c5d64, (0x26b1 bytes) 2024-09-13T05:02:31.474 INFO:teuthology.orchestra.run.smithi053.stdout:1815 writesame 0xe58f0aa thru 0xe5919bf data_size 0x3(0x2916 bytes) 2024-09-13T05:02:31.474 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.474 INFO:teuthology.orchestra.run.smithi053.stdout:1817 punch from 0x17526ba to 0x1755222, (0x2b68 bytes) 2024-09-13T05:02:31.475 INFO:teuthology.orchestra.run.smithi053.stdout:1819 writesame 0x5b5b2d4 thru 0x5b5cac0 data_size 0x23(0x17ed bytes) 2024-09-13T05:02:31.478 INFO:teuthology.orchestra.run.smithi053.stdout:1821 punch from 0x992bdbc to 0x9938433, (0xc677 bytes) 2024-09-13T05:02:31.478 INFO:teuthology.orchestra.run.smithi053.stdout:1822 write 0x1419824 thru 0x14244d0 (0xacad bytes) 2024-09-13T05:02:31.483 INFO:teuthology.orchestra.run.smithi053.stdout:1823 write 0x8653220 thru 0x865c2af (0x9090 bytes) 2024-09-13T05:02:31.490 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:31.490 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:31.490 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:31.491 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.491 INFO:teuthology.orchestra.run.smithi053.stdout:1825 punch from 0x230a224 to 0x23157e1, (0xb5bd bytes) 2024-09-13T05:02:31.497 INFO:teuthology.orchestra.run.smithi053.stdout:1826 writesame 0xee0f5d7 thru 0xee13ec3 data_size 0x31(0x48ed bytes) 2024-09-13T05:02:31.497 INFO:teuthology.orchestra.run.smithi053.stdout:1827 write 0x99f0e65 thru 0x99f3497 (0x2633 bytes) 2024-09-13T05:02:31.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:31.501 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.502 INFO:teuthology.orchestra.run.smithi053.stdout:1829 write 0x1a0289f thru 0x1a127b4 (0xff16 bytes) 2024-09-13T05:02:31.510 INFO:teuthology.orchestra.run.smithi053.stdout:1830 punch from 0xe4d0c92 to 0xe4daabc, (0x9e2a bytes) 2024-09-13T05:02:31.510 INFO:teuthology.orchestra.run.smithi053.stdout:1831 write 0x8838d79 thru 0x8840862 (0x7aea bytes) 2024-09-13T05:02:31.532 INFO:teuthology.orchestra.run.smithi053.stdout:1832 trunc from 0xee13ec4 to 0xa5d916 2024-09-13T05:02:31.547 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:31.546+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:31.558 INFO:teuthology.orchestra.run.smithi053.stdout:1833 punch from 0x249156 to 0x24f6e1, (0x658b bytes) 2024-09-13T05:02:31.558 INFO:teuthology.orchestra.run.smithi053.stdout:1835 read 0x237ace thru 0x240c12 (0x9145 bytes) 2024-09-13T05:02:31.558 INFO:teuthology.orchestra.run.smithi053.stdout:1837 trunc from 0xa5d916 to 0x2d3ae63 2024-09-13T05:02:31.574 INFO:teuthology.orchestra.run.smithi053.stdout:1838 writesame 0x912bb12 thru 0x912f222 data_size 0x25(0x3711 bytes) 2024-09-13T05:02:31.574 INFO:teuthology.orchestra.run.smithi053.stdout:1839 write 0x7b32324 thru 0x7b3c943 (0xa620 bytes) 2024-09-13T05:02:31.581 INFO:teuthology.orchestra.run.smithi053.stdout:1842 read 0x713665b thru 0x714652c (0xfed2 bytes) 2024-09-13T05:02:31.581 INFO:teuthology.orchestra.run.smithi053.stdout:1843 punch from 0x64c40b to 0x655164, (0x8d59 bytes) 2024-09-13T05:02:31.581 INFO:teuthology.orchestra.run.smithi053.stdout:1844 read 0x45ffa43 thru 0x460d011 (0xd5cf bytes) 2024-09-13T05:02:31.581 INFO:teuthology.orchestra.run.smithi053.stdout:1848 write 0x8369b3b thru 0x837778d (0xdc53 bytes) 2024-09-13T05:02:31.599 INFO:teuthology.orchestra.run.smithi053.stdout:1849 write 0xd5b3568 thru 0xd5c1845 (0xe2de bytes) 2024-09-13T05:02:31.608 INFO:teuthology.orchestra.run.smithi053.stdout:1850 write 0xb498000 thru 0xb4a5b2b (0xdb2c bytes) 2024-09-13T05:02:31.617 INFO:teuthology.orchestra.run.smithi053.stdout:1851 write 0xcb49616 thru 0xcb4d3dd (0x3dc8 bytes) 2024-09-13T05:02:31.622 INFO:teuthology.orchestra.run.smithi053.stdout:1852 punch from 0xb9b8d83 to 0xb9bc0e5, (0x3362 bytes) 2024-09-13T05:02:31.622 INFO:teuthology.orchestra.run.smithi053.stdout:1853 read 0x52896ac thru 0x5297b14 (0xe469 bytes) 2024-09-13T05:02:31.622 INFO:teuthology.orchestra.run.smithi053.stdout:1854 punch from 0x8d9414b to 0x8d94ca5, (0xb5a bytes) 2024-09-13T05:02:31.636 INFO:teuthology.orchestra.run.smithi053.stdout:1855 trunc from 0xd5c1846 to 0x5f8514 2024-09-13T05:02:31.644 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:31.644 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:31.644 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:31.648 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:31.647+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:31.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:31.706 INFO:teuthology.orchestra.run.smithi053.stdout:1856 read 0x356fcb thru 0x3577fa (0x830 bytes) 2024-09-13T05:02:31.707 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.707 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.707 INFO:teuthology.orchestra.run.smithi053.stdout:1864 punch from 0x394997 to 0x3a41aa, (0xf813 bytes) 2024-09-13T05:02:31.719 INFO:teuthology.orchestra.run.smithi053.stdout:1865 trunc from 0x5f8514 to 0xb500c3a 2024-09-13T05:02:31.724 INFO:teuthology.orchestra.run.smithi053.stdout:1867 writesame 0x500ad7b thru 0x5013532 data_size 0x65(0x87b8 bytes) 2024-09-13T05:02:31.730 INFO:teuthology.orchestra.run.smithi053.stdout:1868 read 0xac789ac thru 0xac868bc (0xdf11 bytes) 2024-09-13T05:02:31.730 INFO:teuthology.orchestra.run.smithi053.stdout:1870 write 0x65cf295 thru 0x65db558 (0xc2c4 bytes) 2024-09-13T05:02:31.739 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.744 INFO:teuthology.orchestra.run.smithi053.stdout:1872 trunc from 0xb500c3a to 0x78e8289 2024-09-13T05:02:31.748 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:31.748+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:31.755 INFO:teuthology.orchestra.run.smithi053.stdout:1873 trunc from 0x78e8289 to 0xbb4afed 2024-09-13T05:02:31.763 INFO:teuthology.orchestra.run.smithi053.stdout:1874 trunc from 0xbb4afed to 0x8f0cff0 2024-09-13T05:02:31.769 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.769 INFO:teuthology.orchestra.run.smithi053.stdout:1876 read 0x32bf3f5 thru 0x32ced63 (0xf96f bytes) 2024-09-13T05:02:31.777 INFO:teuthology.orchestra.run.smithi053.stdout:1877 write 0xb2b10fe thru 0xb2b5283 (0x4186 bytes) 2024-09-13T05:02:31.780 INFO:teuthology.orchestra.run.smithi053.stdout:1879 read 0x697167 thru 0x69a2ca (0x3164 bytes) 2024-09-13T05:02:31.781 INFO:teuthology.orchestra.run.smithi053.stdout:1880 read 0x55b1028 thru 0x55bf076 (0xe04f bytes) 2024-09-13T05:02:31.791 INFO:teuthology.orchestra.run.smithi053.stdout:1881 trunc from 0xb2b5284 to 0x243d160 2024-09-13T05:02:31.818 INFO:teuthology.orchestra.run.smithi053.stdout:1882 write 0xa98d788 thru 0xa99a65d (0xced6 bytes) 2024-09-13T05:02:31.828 INFO:teuthology.orchestra.run.smithi053.stdout:1883 trunc from 0xa99a65e to 0x909f24e 2024-09-13T05:02:31.834 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:31.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17738564000000001, 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.835 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:02:31.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17736964299999999, 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:31.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:02:31.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17735284700000001, 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.844 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:31.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:02:31.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.177335874, 2024-09-13T05:02:31.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17731948, 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:31.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:02:31.856 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.177302444, 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.857 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:02:31.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.177268861, 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.861 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.862 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:02:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:02:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17724973199999999, 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17717186300000001, 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17711111800000001, 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:02:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:02:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17705090400000001, 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.177033577, 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.177004405, 2024-09-13T05:02:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:02:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17698472100000001, 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:02:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:02:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:02:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.176967288, 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17694964599999999, 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:02:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:02:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17691546999999999, 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:02:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17689012000000001, 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:02:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17687244699999999, 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:02:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.17684824299999999, 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:31.922 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:31.923 INFO:teuthology.orchestra.run.smithi053.stdout:1885 punch from 0x6b02d8e to 0x6b05649, (0x28bb bytes) 2024-09-13T05:02:31.924 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.924 INFO:teuthology.orchestra.run.smithi053.stdout:1887 read 0x628cf52 thru 0x6298f5b (0xc00a bytes) 2024-09-13T05:02:31.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1890 writesame 0xc4a9d23 thru 0xc4b51ae data_size 0xa(0xb48c bytes) 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1891 read 0xb48ffb9 thru 0xb498de7 (0x8e2f bytes) 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1893 read 0x6e6de02 thru 0x6e6de8b (0x8a bytes) 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1894 trunc from 0xc4b51af to 0x3f26918 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1895 read 0x3c43c7e thru 0x3c4733e (0x36c1 bytes) 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1900 write 0x3c9b190 thru 0x3ca0f1f (0x5d90 bytes) 2024-09-13T05:02:31.926 INFO:teuthology.orchestra.run.smithi053.stdout:1901 write 0x91e52ac thru 0x91eb809 (0x655e bytes) 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:1902 writesame 0x339e5c9 thru 0x33a6c21 data_size 0x1(0x8659 bytes) 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:1903 write 0xd6443c4 thru 0xd646d9e (0x29db bytes) 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:1906 write 0x4e5012 thru 0x4ef204 (0xa1f3 bytes) 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:1907 writesame 0x365d8f5 thru 0x366a7b9 data_size 0x1(0xcec5 bytes) 2024-09-13T05:02:31.927 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:31.923+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:31.927 INFO:teuthology.orchestra.run.smithi053.stdout:1908 writesame 0x17ab00b thru 0x17ab0bf data_size 0x1(0xb5 bytes) 2024-09-13T05:02:31.931 INFO:teuthology.orchestra.run.smithi053.stdout:1910 writesame 0xd7ab50d thru 0xd7b5e40 data_size 0x7(0xa934 bytes) 2024-09-13T05:02:31.933 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.934 INFO:teuthology.orchestra.run.smithi053.stdout:1914 writesame 0x89a6f67 thru 0x89b679b data_size 0x1(0xf835 bytes) 2024-09-13T05:02:31.952 INFO:teuthology.orchestra.run.smithi053.stdout:1915 punch from 0xb1fe1d1 to 0xb206a2b, (0x885a bytes) 2024-09-13T05:02:31.952 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.952 INFO:teuthology.orchestra.run.smithi053.stdout:1918 write 0x9d01a76 thru 0x9d04719 (0x2ca4 bytes) 2024-09-13T05:02:31.956 INFO:teuthology.orchestra.run.smithi053.stdout:1919 read 0x4824633 thru 0x482fa6e (0xb43c bytes) 2024-09-13T05:02:31.957 INFO:teuthology.orchestra.run.smithi053.stdout:1921 write 0x8cd80d7 thru 0x8cded17 (0x6c41 bytes) 2024-09-13T05:02:31.966 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:02:31.967 INFO:teuthology.orchestra.run.smithi053.stdout:1922 write 0x254675a thru 0x2551fcc (0xb873 bytes) 2024-09-13T05:02:31.970 INFO:teuthology.orchestra.run.smithi053.stdout:1924 write 0xa75c5ab thru 0xa75cdfd (0x853 bytes) 2024-09-13T05:02:31.973 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.973 INFO:teuthology.orchestra.run.smithi053.stdout:1933 write 0x571cbae thru 0x572542a (0x887d bytes) 2024-09-13T05:02:31.980 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:31.980 INFO:teuthology.orchestra.run.smithi053.stdout:1935 writesame 0x7239265 thru 0x72461d0 data_size 0x24(0xcf6c bytes) 2024-09-13T05:02:31.993 INFO:teuthology.orchestra.run.smithi053.stdout:1936 trunc from 0xd7b5e41 to 0xb0bdbe9 2024-09-13T05:02:32.003 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.003 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.003 INFO:teuthology.orchestra.run.smithi053.stdout:1941 write 0xb5b142 thru 0xb5ccac (0x1b6b bytes) 2024-09-13T05:02:32.007 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.007 INFO:teuthology.orchestra.run.smithi053.stdout:1943 writesame 0x128f65e thru 0x129ad35 data_size 0x4(0xb6d8 bytes) 2024-09-13T05:02:32.024 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:32.023+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:32.027 INFO:teuthology.orchestra.run.smithi053.stdout:1945 writesame 0xcd59728 thru 0xcd65ab5 data_size 0x1(0xc38e bytes) 2024-09-13T05:02:32.027 INFO:teuthology.orchestra.run.smithi053.stdout:1946 writesame 0x4666620 thru 0x4673b3c data_size 0x1(0xd51d bytes) 2024-09-13T05:02:32.040 INFO:teuthology.orchestra.run.smithi053.stdout:1947 write 0x7719187 thru 0x771b0de (0x1f58 bytes) 2024-09-13T05:02:32.044 INFO:teuthology.orchestra.run.smithi053.stdout:1948 write 0xc99574e thru 0xc99a49a (0x4d4d bytes) 2024-09-13T05:02:32.053 INFO:teuthology.orchestra.run.smithi053.stdout:1949 trunc from 0xcd65ab6 to 0xac8926f 2024-09-13T05:02:32.062 INFO:teuthology.orchestra.run.smithi053.stdout:1950 write 0x52021e thru 0x526e98 (0x6c7b bytes) 2024-09-13T05:02:32.065 INFO:teuthology.orchestra.run.smithi053.stdout:1951 trunc from 0xac8926f to 0xaef2e61 2024-09-13T05:02:32.069 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.069 INFO:teuthology.orchestra.run.smithi053.stdout:1955 write 0x1c10560 thru 0x1c1e7ef (0xe290 bytes) 2024-09-13T05:02:32.079 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:32.079 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:32.079 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:32.079 INFO:teuthology.orchestra.run.smithi053.stdout:1956 write 0x74922d0 thru 0x749b954 (0x9685 bytes) 2024-09-13T05:02:32.086 INFO:teuthology.orchestra.run.smithi053.stdout:1957 read 0xdc4e4b thru 0xdd4584 (0xf73a bytes) 2024-09-13T05:02:32.087 INFO:teuthology.orchestra.run.smithi053.stdout:1958 write 0x9dfb356 thru 0x9e09be5 (0xe890 bytes) 2024-09-13T05:02:32.092 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:32.097 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.098 INFO:teuthology.orchestra.run.smithi053.stdout:1960 write 0x6992bcd thru 0x69983ac (0x57e0 bytes) 2024-09-13T05:02:32.103 INFO:teuthology.orchestra.run.smithi053.stdout:1961 punch from 0x4ec0898 to 0x4ecd3b0, (0xcb18 bytes) 2024-09-13T05:02:32.104 INFO:teuthology.orchestra.run.smithi053.stdout:1964 read 0x4334ab2 thru 0x4334c2b (0x17a bytes) 2024-09-13T05:02:32.104 INFO:teuthology.orchestra.run.smithi053.stdout:1965 read 0x2b2d071 thru 0x2b3792f (0xa8bf bytes) 2024-09-13T05:02:32.104 INFO:teuthology.orchestra.run.smithi053.stdout:1966 read 0x690fe8d thru 0x69101f9 (0x36d bytes) 2024-09-13T05:02:32.104 INFO:teuthology.orchestra.run.smithi053.stdout:1967 writesame 0x6c67c7a thru 0x6c70f4a data_size 0x1(0x92d1 bytes) 2024-09-13T05:02:32.112 INFO:teuthology.orchestra.run.smithi053.stdout:1968 read 0x32fa7a0 thru 0x3301ba1 (0x7402 bytes) 2024-09-13T05:02:32.112 INFO:teuthology.orchestra.run.smithi053.stdout:1969 read 0xade1a9e thru 0xadea6e7 (0x8c4a bytes) 2024-09-13T05:02:32.112 INFO:teuthology.orchestra.run.smithi053.stdout:1970 read 0x5dbc6d4 thru 0x5dc55a8 (0x8ed5 bytes) 2024-09-13T05:02:32.121 INFO:teuthology.orchestra.run.smithi053.stdout:1971 write 0xd3ddd77 thru 0xd3e5291 (0x751b bytes) 2024-09-13T05:02:32.125 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:32.125+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:32.130 INFO:teuthology.orchestra.run.smithi053.stdout:1973 write 0x989aefe thru 0x98a9ba0 (0xeca3 bytes) 2024-09-13T05:02:32.140 INFO:teuthology.orchestra.run.smithi053.stdout:1974 write 0x2331adf thru 0x233646e (0x4990 bytes) 2024-09-13T05:02:32.144 INFO:teuthology.orchestra.run.smithi053.stdout:1977 write 0xb07b20e thru 0xb083788 (0x857b bytes) 2024-09-13T05:02:32.151 INFO:teuthology.orchestra.run.smithi053.stdout:1979 read 0x9fb3c8a thru 0x9fc19e7 (0xdd5e bytes) 2024-09-13T05:02:32.151 INFO:teuthology.orchestra.run.smithi053.stdout:1980 punch from 0x1967d0f to 0x196bc7e, (0x3f6f bytes) 2024-09-13T05:02:32.152 INFO:teuthology.orchestra.run.smithi053.stdout:1981 read 0x915c862 thru 0x916c3df (0xfb7e bytes) 2024-09-13T05:02:32.152 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.152 INFO:teuthology.orchestra.run.smithi053.stdout:1984 write 0x9577063 thru 0x9583fcd (0xcf6b bytes) 2024-09-13T05:02:32.161 INFO:teuthology.orchestra.run.smithi053.stdout:1985 punch from 0x8418b51 to 0x8428514, (0xf9c3 bytes) 2024-09-13T05:02:32.171 INFO:teuthology.orchestra.run.smithi053.stdout:1986 trunc from 0xd3e5292 to 0x4e116af 2024-09-13T05:02:32.213 INFO:teuthology.orchestra.run.smithi053.stdout:1988 writesame 0xc78fdcf thru 0xc79385b data_size 0x1(0x3a8d bytes) 2024-09-13T05:02:32.219 INFO:teuthology.orchestra.run.smithi053.stdout:1989 trunc from 0xc79385c to 0x6e09ea6 2024-09-13T05:02:32.226 INFO:teuthology.orchestra.run.smithi053.stdout:1990 punch from 0xd7d51a to 0xd8b7b4, (0xe29a bytes) 2024-09-13T05:02:32.226 INFO:teuthology.orchestra.run.smithi053.stdout:1991 read 0x25605c4 thru 0x256b2bc (0xacf9 bytes) 2024-09-13T05:02:32.226 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:32.225+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:32.226 INFO:teuthology.orchestra.run.smithi053.stdout:1992 write 0x394cad4 thru 0x394e951 (0x1e7e bytes) 2024-09-13T05:02:32.229 INFO:teuthology.orchestra.run.smithi053.stdout:1993 write 0x14917d4 thru 0x149b02d (0x985a bytes) 2024-09-13T05:02:32.239 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:32.240 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:32.240 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:32.240 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:32.240 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:32.248 INFO:teuthology.orchestra.run.smithi053.stdout:1994 write 0xd7878f5 thru 0xd78d0b8 (0x57c4 bytes) 2024-09-13T05:02:32.251 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:32.255 INFO:teuthology.orchestra.run.smithi053.stdout:1997 writesame 0x5cf7f1b thru 0x5d067bb data_size 0x27(0xe8a1 bytes) 2024-09-13T05:02:32.266 INFO:teuthology.orchestra.run.smithi053.stdout:2000 read 0xa91ec70 thru 0xa92d08c (0xe41d bytes) 2024-09-13T05:02:32.266 INFO:teuthology.orchestra.run.smithi053.stdout:2001 write 0x3ff03e9 thru 0x3fff658 (0xf270 bytes) 2024-09-13T05:02:32.276 INFO:teuthology.orchestra.run.smithi053.stdout:2003 read 0xb9f0455 thru 0xb9f1a8f (0x163b bytes) 2024-09-13T05:02:32.276 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.276 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:32.276 INFO:teuthology.orchestra.run.smithi053.stdout:2006 write 0x9b111eb thru 0x9b1c31b (0xb131 bytes) 2024-09-13T05:02:32.285 INFO:teuthology.orchestra.run.smithi053.stdout:2007 clone 15 order 19 su 65536 sc 12 2024-09-13T05:02:32.327 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:32.326+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32275 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5882403920000003, 2024-09-13T05:02:32.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010461302, 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32275 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780089+0000", 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2010000000000002e-06 2024-09-13T05:02:32.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780095+0000", 2024-09-13T05:02:32.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8019999999999997e-06 2024-09-13T05:02:32.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780099+0000", 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3229999999999999e-06 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790415+0000", 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010315649 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790475+0000", 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9843999999999998e-05 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790549+0000", 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3883999999999999e-05 2024-09-13T05:02:32.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32276 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5882246230000003, 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010619966, 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32276 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780105+0000", 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.159e-06 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780111+0000", 2024-09-13T05:02:32.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5639999999999996e-06 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780118+0000", 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790585+0000", 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010466197 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790648+0000", 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3927e-05 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790723+0000", 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4635999999999995e-05 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32277 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8454144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5882056520000001, 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010784396999999999, 2024-09-13T05:02:32.384 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32277 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780124+0000", 2024-09-13T05:02:32.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119e-06 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780129+0000", 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5799999999999999e-06 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780134+0000", 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4750000000000004e-06 2024-09-13T05:02:32.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790750+0000", 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010616082000000001 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790824+0000", 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4031999999999999e-05 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790907+0000", 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2458000000000002e-05 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32278 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8683520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5881626779999998, 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010917582, 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.389 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32278 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780164+0000", 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780169+0000", 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8300000000000003e-06 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780179+0000", 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0164e-05 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780186+0000", 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4799999999999998e-06 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790933+0000", 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010747358 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791001+0000", 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7390000000000004e-05 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791083+0000", 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2177999999999998e-05 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32279 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8912896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5881366239999997, 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011066273999999999, 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32279 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780193+0000", 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780200+0000", 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3709999999999996e-06 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780205+0000", 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-06 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791108+0000", 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010903375 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791175+0000", 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6451999999999993e-05 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791257+0000", 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2645999999999994e-05 2024-09-13T05:02:32.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.399 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32317 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8945664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5879804100000001, 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010478401, 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32317 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780350+0000", 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9790000000000002e-06 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780357+0000", 2024-09-13T05:02:32.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2350000000000002e-06 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780362+0000", 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790666+0000", 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010303911000000001 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790735+0000", 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8396e-05 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790826+0000", 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0983999999999995e-05 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32318 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9175040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.587960689, 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010623140999999999, 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32318 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.405 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780369+0000", 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5620000000000002e-06 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780375+0000", 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780380+0000", 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6879999999999998e-06 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790855+0000", 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010474957 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790920+0000", 2024-09-13T05:02:32.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5030999999999995e-05 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790990+0000", 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0384999999999998e-05 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32337 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5879310100000001, 2024-09-13T05:02:32.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011667398000000001, 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32337 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780396+0000", 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780399+0000", 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9849999999999998e-06 2024-09-13T05:02:32.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780406+0000", 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780415+0000", 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6799999999999999e-06 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791286+0000", 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010870869999999999 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791955+0000", 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066933100000000001 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792064+0000", 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010907499999999999 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32338 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7798784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.58790795, 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011830083, 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32338 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780419+0000", 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780422+0000", 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.339e-06 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780428+0000", 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0730000000000001e-06 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780433+0000", 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7010000000000004e-06 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792095+0000", 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011662529 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792164+0000", 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9346999999999999e-05 2024-09-13T05:02:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792250+0000", 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5504e-05 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32339 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8028160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5878909810000001, 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011996256, 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32339 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780436+0000", 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780439+0000", 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.198e-06 2024-09-13T05:02:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780444+0000", 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.806e-06 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780451+0000", 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792275+0000", 2024-09-13T05:02:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011824573999999999 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792339+0000", 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3628999999999997e-05 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792433+0000", 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4029000000000004e-05 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32340 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8257536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5878729280000003, 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012155473999999999, 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32340 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780457+0000", 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780462+0000", 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7490000000000004e-06 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780467+0000", 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5970000000000002e-06 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792461+0000", 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011993668000000001 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792529+0000", 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8249000000000003e-05 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792610+0000", 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1451000000000003e-05 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32341 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8486912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5878565739999999, 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012301632999999999, 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32341 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780473+0000", 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3269999999999999e-06 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780481+0000", 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7970000000000001e-06 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780485+0000", 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.408e-06 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792636+0000", 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012150879 2024-09-13T05:02:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792699+0000", 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2842000000000006e-05 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792773+0000", 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000001e-05 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32342 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8716288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.587837661, 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012452926, 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32342 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780493+0000", 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9979999999999999e-06 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780499+0000", 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-06 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780504+0000", 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6909999999999996e-06 2024-09-13T05:02:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792805+0000", 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012301288 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792868+0000", 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3202999999999998e-05 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792943+0000", 2024-09-13T05:02:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4592999999999995e-05 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32343 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8945664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5878188099999999, 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012601064, 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32343 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780511+0000", 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.215e-06 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780516+0000", 2024-09-13T05:02:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6380000000000001e-06 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780521+0000", 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792968+0000", 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012446262 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793031+0000", 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3609999999999996e-05 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793110+0000", 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8910000000000002e-05 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32344 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 9175040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.587802259, 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012769502, 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32344 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780525+0000", 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780527+0000", 2024-09-13T05:02:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3649999999999998e-06 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780534+0000", 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5980000000000002e-06 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780538+0000", 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.194e-06 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793139+0000", 2024-09-13T05:02:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012600731 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793210+0000", 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1131999999999996e-05 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793295+0000", 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4938000000000006e-05 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32379 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8749056~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5867621429999996, 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010495398, 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32379 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781565+0000", 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781568+0000", 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781574+0000", 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2929999999999997e-06 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781579+0000", 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0340000000000004e-06 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791907+0000", 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010328027 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791977+0000", 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9728999999999996e-05 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792061+0000", 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4332000000000005e-05 2024-09-13T05:02:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32380 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8978432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.586744328, 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010658737, 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32380 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781585+0000", 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0279999999999999e-06 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781589+0000", 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:02:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781591+0000", 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6120000000000001e-06 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792087+0000", 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010496152 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792161+0000", 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3473999999999995e-05 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792242+0000", 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1403999999999995e-05 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32381 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9207808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5867344719999998, 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012026156, 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32381 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781594+0000", 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3850000000000001e-06 2024-09-13T05:02:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781597+0000", 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9579999999999999e-06 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781600+0000", 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4169999999999999e-06 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792270+0000", 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010670059000000001 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792339+0000", 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8861000000000004e-05 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793619+0000", 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001280712 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32382 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9437184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5867254410000005, 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012273719000000001, 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32382 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781604+0000", 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.905e-06 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781610+0000", 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3799999999999999e-06 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781614+0000", 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5870000000000001e-06 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793687+0000", 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012073084 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793763+0000", 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6074000000000004e-05 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793876+0000", 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011301199999999999 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:40343 3.4 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.936337+0000", 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.43199045400000002, 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011182364, 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 40343 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936337+0000", 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936337+0000", 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936336+0000", 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936345+0000", 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9409999999999996e-06 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936362+0000", 2024-09-13T05:02:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7144000000000001e-05 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936372+0000", 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7950000000000002e-06 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936436+0000", 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4688000000000001e-05 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936587+0000", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015060400000000001 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.937505+0000", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091796800000000002 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947390+0000", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098854700000000004 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947432+0000", 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1678999999999999e-05 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947479+0000", 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7119000000000001e-05 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947520+0000", 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0646999999999999e-05 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.468 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:32.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:32.470 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:32.469+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:32.569 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:32.569+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:32.617 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:32.617 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:32.617 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:32.617 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:32.626 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:32.670 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:32.670+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:32.771 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:32.771+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:32.772 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:32.772 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:32.772 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:32.780 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:32.871 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:32.871+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:32.888 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:32.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:9144 3.4 e28/19 3:21949514:::rbd_data.117c6c62fa22.0000000000000052:head v 28'73, mlcod=27'71)", 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.974798188999999, 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001561832, 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912419+0000", 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8749999999999999e-06 2024-09-13T05:02:32.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912432+0000", 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2373e-05 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912440+0000", 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2889999999999998e-06 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913219+0000", 2024-09-13T05:02:32.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077888000000000002 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913240+0000", 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0672000000000001e-05 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913941+0000", 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070157300000000004 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913977+0000", 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5472000000000002e-05 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.909149097, 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.893 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:02:32.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10094 3.4 e29/19 3:24263961:::rbd_data.117c6c62fa22.000000000000000b:head v 29'95, mlcod=28'92)", 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.906077688, 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001767984, 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981152+0000", 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6986999999999998e-05 2024-09-13T05:02:32.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981166+0000", 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4331000000000001e-05 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981177+0000", 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0858e-05 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981233+0000", 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5813000000000002e-05 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981253+0000", 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9707999999999998e-05 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982869+0000", 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001615932 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982903+0000", 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4626999999999999e-05 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10121 3.4 e29/19 3:2672cf5a:::rbd_data.117c6c62fa22.0000000000000029:head v 29'97, mlcod=28'93)", 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.989656+0000", 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.897556771, 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001391357, 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.902 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989656+0000", 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989656+0000", 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989655+0000", 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989663+0000", 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8760000000000005e-06 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989677+0000", 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.398e-05 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989684+0000", 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8600000000000004e-06 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989733+0000", 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8950999999999999e-05 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989755+0000", 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2008999999999999e-05 2024-09-13T05:02:32.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.991015+0000", 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012602970000000001 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.991048+0000", 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2249000000000002e-05 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10141 3.4 e29/19 3:24508b78:::rbd_data.117c6c62fa22.000000000000003d:head v 29'101, mlcod=29'97)", 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.892334975000001, 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001501142, 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994877+0000", 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994885+0000", 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-09-13T05:02:32.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994900+0000", 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4698999999999999e-05 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994907+0000", 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2570000000000001e-06 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994952+0000", 2024-09-13T05:02:32.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4885000000000001e-05 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994971+0000", 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9093000000000002e-05 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996346+0000", 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001375287 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996379+0000", 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2700999999999997e-05 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10143 3.0 e29/19 3:0297533c:::rbd_data.117c6c62fa22.000000000000003f:head v 29'99, mlcod=28'93)", 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.891842808, 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001524173, 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995375+0000", 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8340000000000004e-06 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995387+0000", 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1046000000000001e-05 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995393+0000", 2024-09-13T05:02:32.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995441+0000", 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7786e-05 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995460+0000", 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9154e-05 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996857+0000", 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013971719999999999 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996894+0000", 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7357999999999997e-05 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:19907 3.9 e31/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 31'115, mlcod=31'112)", 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:32.915 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 10.440368938000001, 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001592883, 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446843+0000", 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446850+0000", 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3450000000000002e-06 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446860+0000", 2024-09-13T05:02:32.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446867+0000", 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3570000000000003e-06 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446923+0000", 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6242000000000001e-05 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446940+0000", 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7209000000000001e-05 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448406+0000", 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001465383 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448437+0000", 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1372999999999999e-05 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:20095 3.9 e32/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 32'140, mlcod=32'137)", 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6913851760000007, 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014673279999999999, 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195827+0000", 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195846+0000", 2024-09-13T05:02:32.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8810000000000001e-05 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195860+0000", 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3835000000000001e-05 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195868+0000", 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2300000000000008e-06 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195910+0000", 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1690000000000002e-05 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195938+0000", 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8829000000000001e-05 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.197231+0000", 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001292201 2024-09-13T05:02:32.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.197295+0000", 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4498000000000005e-05 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:20115 3.0 e32/19 3:07e79267:::rbd_data.117c3521aa89.000000000000000a:head v 32'193, mlcod=31'190)", 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.6845944429999999, 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015367919999999999, 2024-09-13T05:02:32.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202618+0000", 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202626+0000", 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0700000000000007e-06 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202636+0000", 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0227000000000001e-05 2024-09-13T05:02:32.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202645+0000", 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8040000000000007e-06 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202687+0000", 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2360000000000001e-05 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202706+0000", 2024-09-13T05:02:32.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8933e-05 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.204117+0000", 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014105089999999999 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.204155+0000", 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8544999999999998e-05 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.0837650419999996, 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40302 3.4 e37/19 3:251ae013:::rbd_data.117c4a06f7bc.0000000000000019:head v 37'208, mlcod=37'206)", 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.909225+0000", 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.97798769299999999, 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001368874, 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909225+0000", 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909225+0000", 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909224+0000", 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909240+0000", 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6334e-05 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909256+0000", 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5948e-05 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909262+0000", 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.443e-06 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909313+0000", 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0980999999999999e-05 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.909340+0000", 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6301000000000001e-05 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.910570+0000", 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001230718 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.910594+0000", 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3754999999999999e-05 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.94269600799999997, 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:02:32.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:02:32.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40489 3.4 e37/19 3:234d8752:::rbd_data.117c4a06f7bc.000000000000001a:head v 37'216, mlcod=37'214)", 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.82335762599999995, 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014028300000000001, 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063866+0000", 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1114000000000001e-05 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063873+0000", 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2180000000000002e-06 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063879+0000", 2024-09-13T05:02:32.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.064215+0000", 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033607 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.064232+0000", 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.768e-05 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.065228+0000", 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099559299999999991 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.065258+0000", 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0261999999999999e-05 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:02:32.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.71117535799999998, 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:32.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:02:32.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.71019968499999997, 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:02:32.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.70987880800000003, 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.955 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.70779085100000005, 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:02:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:02:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40675 3.4 e37/19 3:25c10db2:::rbd_data.117c4a06f7bc.000000000000024b:head v 37'226, mlcod=37'224)", 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.70408423899999995, 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001577501, 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183128+0000", 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183134+0000", 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.387e-06 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183149+0000", 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4664000000000001e-05 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183156+0000", 2024-09-13T05:02:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9700000000000002e-06 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183436+0000", 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027994199999999998 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183455+0000", 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9287999999999999e-05 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184676+0000", 2024-09-13T05:02:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001220308 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184706+0000", 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639000000000003e-05 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40676 3.9 e37/19 3:90539305:::rbd_data.117c4a06f7bc.0000000000000253:head v 37'189, mlcod=37'186)", 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.183259+0000", 2024-09-13T05:02:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.703954046, 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013780439999999999, 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183259+0000", 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183259+0000", 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183258+0000", 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183264+0000", 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7660000000000004e-06 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183278+0000", 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3424000000000001e-05 2024-09-13T05:02:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183284+0000", 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.567e-06 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183335+0000", 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0883000000000001e-05 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183354+0000", 2024-09-13T05:02:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9278000000000001e-05 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184600+0000", 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001245859 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184637+0000", 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6869999999999998e-05 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40683 3.4 e37/19 3:2071a6c5:::rbd_data.117c4a06f7bc.000000000000025d:head v 37'227, mlcod=37'225)", 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.70192109599999997, 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001476371, 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185291+0000", 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185298+0000", 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185313+0000", 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5406000000000001e-05 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185326+0000", 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2313999999999999e-05 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185431+0000", 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010521799999999999 2024-09-13T05:02:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185450+0000", 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9315000000000001e-05 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.186710+0000", 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012597730000000001 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.186768+0000", 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8236000000000003e-05 2024-09-13T05:02:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:32.977 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:32.978 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:32.979 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:02:32.979 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:32.977+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:33.032 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone14 from 0xd78d0b9 (overlap 0x5f8514) to 0x42dd27 2024-09-13T05:02:33.079 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:33.078+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:33.142 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-13T05:02:33.145 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:33.145 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:33.145 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:33.145 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:33.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:33.158 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:33.180 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:33.180+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:33.242 INFO:teuthology.orchestra.run.smithi053.stdout:2010 write 0xa8625ac thru 0xa862cb4 (0x709 bytes) 2024-09-13T05:02:33.250 INFO:teuthology.orchestra.run.smithi053.stdout:2011 read 0x749dd3 thru 0x751830 (0x7a5e bytes) 2024-09-13T05:02:33.250 INFO:teuthology.orchestra.run.smithi053.stdout:2012 write 0x22bc04f thru 0x22c3403 (0x73b5 bytes) 2024-09-13T05:02:33.256 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.256 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.256 INFO:teuthology.orchestra.run.smithi053.stdout:2015 writesame 0x4d0bc8e thru 0x4d12ecc data_size 0x1(0x723f bytes) 2024-09-13T05:02:33.269 INFO:teuthology.orchestra.run.smithi053.stdout:2017 punch from 0x26037e1 to 0x2604318, (0xb37 bytes) 2024-09-13T05:02:33.269 INFO:teuthology.orchestra.run.smithi053.stdout:2018 read 0x47fc638 thru 0x47ff219 (0x2be2 bytes) 2024-09-13T05:02:33.269 INFO:teuthology.orchestra.run.smithi053.stdout:2019 write 0x3225a5f thru 0x323029b (0xa83d bytes) 2024-09-13T05:02:33.276 INFO:teuthology.orchestra.run.smithi053.stdout:2020 write 0x47992fe thru 0x479b528 (0x222b bytes) 2024-09-13T05:02:33.281 INFO:teuthology.orchestra.run.smithi053.stdout:2021 write 0x61900df thru 0x619e5dd (0xe4ff bytes) 2024-09-13T05:02:33.281 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:33.280+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:33.285 INFO:teuthology.orchestra.run.smithi053.stdout:2022 write 0x403c258 thru 0x40489fa (0xc7a3 bytes) 2024-09-13T05:02:33.292 INFO:teuthology.orchestra.run.smithi053.stdout:2023 writesame 0xcb8fdc thru 0xcc7543 data_size 0xc(0xe568 bytes) 2024-09-13T05:02:33.298 INFO:teuthology.orchestra.run.smithi053.stdout:2024 read 0x9caf77c thru 0x9cb7d0c (0x8591 bytes) 2024-09-13T05:02:33.299 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.299 INFO:teuthology.orchestra.run.smithi053.stdout:2027 read 0x840f90c thru 0x84189e8 (0x90dd bytes) 2024-09-13T05:02:33.299 INFO:teuthology.orchestra.run.smithi053.stdout:2028 read 0x607a5ee thru 0x607dc2e (0x3641 bytes) 2024-09-13T05:02:33.299 INFO:teuthology.orchestra.run.smithi053.stdout:2029 read 0xb7b8559 thru 0xb7c34e6 (0xaf8e bytes) 2024-09-13T05:02:33.299 INFO:teuthology.orchestra.run.smithi053.stdout:2031 writesame 0x75f71f thru 0x76d052 data_size 0x2(0xd934 bytes) 2024-09-13T05:02:33.305 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:33.306 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:33.315 INFO:teuthology.orchestra.run.smithi053.stdout:2032 trunc from 0xd78d0b9 to 0xa0f9d1a 2024-09-13T05:02:33.328 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:33.328 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:33.328 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:33.329 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:33.329 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:33.329 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:33.329 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:33.343 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:33.346 INFO:teuthology.orchestra.run.smithi053.stdout:2034 trunc from 0xa0f9d1a to 0x9d35753 2024-09-13T05:02:33.353 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:33.354 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:33.354 INFO:teuthology.orchestra.run.smithi053.stdout:2035 punch from 0x1004302 to 0x1012a92, (0xe790 bytes) 2024-09-13T05:02:33.354 INFO:teuthology.orchestra.run.smithi053.stdout:2037 write 0x32a43af thru 0x32a4fd6 (0xc28 bytes) 2024-09-13T05:02:33.360 INFO:teuthology.orchestra.run.smithi053.stdout:2038 writesame 0x74db476 thru 0x74ddd1a data_size 0x1(0x28a5 bytes) 2024-09-13T05:02:33.367 INFO:teuthology.orchestra.run.smithi053.stdout:2039 read 0x24452ac thru 0x2446375 (0x10ca bytes) 2024-09-13T05:02:33.367 INFO:teuthology.orchestra.run.smithi053.stdout:2040 writesame 0x7cb9d3a thru 0x7cbbe2f data_size 0x1(0x20f6 bytes) 2024-09-13T05:02:33.374 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:33.374 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:33.374 INFO:teuthology.orchestra.run.smithi053.stdout:2041 read 0xd2e0f0 thru 0xd3a825 (0xc736 bytes) 2024-09-13T05:02:33.375 INFO:teuthology.orchestra.run.smithi053.stdout:2042 writesame 0x79fa717 thru 0x79fe035 data_size 0x1(0x391f bytes) 2024-09-13T05:02:33.382 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:33.381+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:33.383 INFO:teuthology.orchestra.run.smithi053.stdout:2044 writesame 0x72ab747 thru 0x72b9b22 data_size 0x6(0xe3dc bytes) 2024-09-13T05:02:33.394 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:33.394 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:33.401 INFO:teuthology.orchestra.run.smithi053.stdout:2045 writesame 0xcf6f74d thru 0xcf7de1d data_size 0x1(0xe6d1 bytes) 2024-09-13T05:02:33.401 INFO:teuthology.orchestra.run.smithi053.stdout:2046 read 0x707dc22 thru 0x70878aa (0x9c89 bytes) 2024-09-13T05:02:33.407 INFO:teuthology.orchestra.run.smithi053.stdout:2047 trunc from 0xcf7de1e to 0x7adf946 2024-09-13T05:02:33.414 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:33.414 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:33.432 INFO:teuthology.orchestra.run.smithi053.stdout:2048 read 0x1ce3457 thru 0x1ce7f22 (0x4acc bytes) 2024-09-13T05:02:33.438 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:33.438 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:33.445 INFO:teuthology.orchestra.run.smithi053.stdout:2049 write 0xe4aa1ef thru 0xe4b36f2 (0x9504 bytes) 2024-09-13T05:02:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:2050 punch from 0xd503f14 to 0xd50e82a, (0xa916 bytes) 2024-09-13T05:02:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:2054 punch from 0x7bc5e9b to 0x7bc7f8d, (0x20f2 bytes) 2024-09-13T05:02:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:2056 read 0x3bf16c6 thru 0x3bf3faf (0x28ea bytes) 2024-09-13T05:02:33.450 INFO:teuthology.orchestra.run.smithi053.stdout:2058 writesame 0x48c36aa thru 0x48d0916 data_size 0x1(0xd26d bytes) 2024-09-13T05:02:33.458 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:33.459 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:33.475 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:33.475 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:33.477 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.477 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.478 INFO:teuthology.orchestra.run.smithi053.stdout:2063 write 0x628e1bc thru 0x628e1db (0x20 bytes) 2024-09-13T05:02:33.481 INFO:teuthology.orchestra.run.smithi053.stdout:2064 punch from 0xda50f41 to 0xda5f62c, (0xe6eb bytes) 2024-09-13T05:02:33.481 INFO:teuthology.orchestra.run.smithi053.stdout:2066 read 0xa017b54 thru 0xa01d20e (0x56bb bytes) 2024-09-13T05:02:33.481 INFO:teuthology.orchestra.run.smithi053.stdout:2067 punch from 0x5c2eadc to 0x5c35f6d, (0x7491 bytes) 2024-09-13T05:02:33.481 INFO:teuthology.orchestra.run.smithi053.stdout:2068 writesame 0x139b538 thru 0x139ff78 data_size 0x1(0x4a41 bytes) 2024-09-13T05:02:33.482 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:33.482+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:33.486 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:33.493 INFO:teuthology.orchestra.run.smithi053.stdout:2069 read 0x9e69426 thru 0x9e7037c (0x6f57 bytes) 2024-09-13T05:02:33.493 INFO:teuthology.orchestra.run.smithi053.stdout:2070 writesame 0xa5c3ca1 thru 0xa5c57db data_size 0x1(0x1b3b bytes) 2024-09-13T05:02:33.498 INFO:teuthology.orchestra.run.smithi053.stdout:2072 read 0xd94786b thru 0xd956431 (0xebc7 bytes) 2024-09-13T05:02:33.498 INFO:teuthology.orchestra.run.smithi053.stdout:2073 read 0xd588a25 thru 0xd58ce17 (0x43f3 bytes) 2024-09-13T05:02:33.499 INFO:teuthology.orchestra.run.smithi053.stdout:2074 write 0xb827d26 thru 0xb82fbae (0x7e89 bytes) 2024-09-13T05:02:33.501 INFO:teuthology.orchestra.run.smithi053.stdout:2075 write 0xb8ecc1b thru 0xb8f2c3c (0x6022 bytes) 2024-09-13T05:02:33.505 INFO:teuthology.orchestra.run.smithi053.stdout:2076 read 0x2b3f4b3 thru 0x2b4b06f (0xbbbd bytes) 2024-09-13T05:02:33.505 INFO:teuthology.orchestra.run.smithi053.stdout:2077 writesame 0xb06b8a thru 0xb0d9fd data_size 0x2(0x6e74 bytes) 2024-09-13T05:02:33.514 INFO:teuthology.orchestra.run.smithi053.stdout:2079 read 0x6885bb0 thru 0x6891232 (0xb683 bytes) 2024-09-13T05:02:33.514 INFO:teuthology.orchestra.run.smithi053.stdout:2081 write 0xb67d673 thru 0xb684a95 (0x7423 bytes) 2024-09-13T05:02:33.518 INFO:teuthology.orchestra.run.smithi053.stdout:2082 punch from 0x2462c3c to 0x24713b8, (0xe77c bytes) 2024-09-13T05:02:33.518 INFO:teuthology.orchestra.run.smithi053.stdout:2083 punch from 0xdc07e11 to 0xdc0e9be, (0x6bad bytes) 2024-09-13T05:02:33.523 INFO:teuthology.orchestra.run.smithi053.stdout:2084 writesame 0xeba1d7f thru 0xebaec5a data_size 0x12(0xcedc bytes) 2024-09-13T05:02:33.523 INFO:teuthology.orchestra.run.smithi053.stdout:2085 read 0x7a15e56 thru 0x7a1e576 (0x8721 bytes) 2024-09-13T05:02:33.524 INFO:teuthology.orchestra.run.smithi053.stdout:2086 writesame 0x3676ea6 thru 0x3678493 data_size 0x7(0x15ee bytes) 2024-09-13T05:02:33.527 INFO:teuthology.orchestra.run.smithi053.stdout:2087 write 0x5e3691d thru 0x5e44575 (0xdc59 bytes) 2024-09-13T05:02:33.547 INFO:teuthology.orchestra.run.smithi053.stdout:2088 trunc from 0xebaec5b to 0x2470e51 2024-09-13T05:02:33.583 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:33.583+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:33.586 INFO:teuthology.orchestra.run.smithi053.stdout:2089 write 0x76e37a7 thru 0x76eb79e (0x7ff8 bytes) 2024-09-13T05:02:33.594 INFO:teuthology.orchestra.run.smithi053.stdout:2090 write 0x8343be2 thru 0x83484ba (0x48d9 bytes) 2024-09-13T05:02:33.597 INFO:teuthology.orchestra.run.smithi053.stdout:2092 write 0x5b8191 thru 0x5bf327 (0x7197 bytes) 2024-09-13T05:02:33.601 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:33.601 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:33.605 INFO:teuthology.orchestra.run.smithi053.stdout:2093 writesame 0x4049d72 thru 0x4058de9 data_size 0x36(0xf078 bytes) 2024-09-13T05:02:33.610 INFO:teuthology.orchestra.run.smithi053.stdout:2094 trunc from 0x83484bb to 0x8a4b6db 2024-09-13T05:02:33.611 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:33.614 INFO:teuthology.orchestra.run.smithi053.stdout:2095 read 0x830817b thru 0x8308c75 (0xafb bytes) 2024-09-13T05:02:33.615 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.615 INFO:teuthology.orchestra.run.smithi053.stdout:2098 read 0x23bf09d thru 0x23c611b (0x707f bytes) 2024-09-13T05:02:33.615 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.617 INFO:teuthology.orchestra.run.smithi053.stdout:2101 trunc from 0x8a4b6db to 0x735d165 2024-09-13T05:02:33.635 INFO:teuthology.orchestra.run.smithi053.stdout:2102 write 0x9f87060 thru 0x9f875af (0x550 bytes) 2024-09-13T05:02:33.637 INFO:teuthology.orchestra.run.smithi053.stdout:2104 read 0x810ce6 thru 0x819279 (0x8594 bytes) 2024-09-13T05:02:33.638 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.638 INFO:teuthology.orchestra.run.smithi053.stdout:2106 read 0x931767e thru 0x9317b30 (0x4b3 bytes) 2024-09-13T05:02:33.638 INFO:teuthology.orchestra.run.smithi053.stdout:2107 write 0xcdf3dc thru 0xce3238 (0x3e5d bytes) 2024-09-13T05:02:33.643 INFO:teuthology.orchestra.run.smithi053.stdout:2108 writesame 0x2d175d6 thru 0x2d1fd0f data_size 0x13(0x873a bytes) 2024-09-13T05:02:33.646 INFO:teuthology.orchestra.run.smithi053.stdout:2109 read 0x1a3e747 thru 0x1a41e75 (0x372f bytes) 2024-09-13T05:02:33.646 INFO:teuthology.orchestra.run.smithi053.stdout:2111 read 0x7aed240 thru 0x7af732a (0xa0eb bytes) 2024-09-13T05:02:33.646 INFO:teuthology.orchestra.run.smithi053.stdout:2112 read 0x749e887 thru 0x74ac1d4 (0xd94e bytes) 2024-09-13T05:02:33.647 INFO:teuthology.orchestra.run.smithi053.stdout:2114 write 0x12fa05f thru 0x12fb5f4 (0x1596 bytes) 2024-09-13T05:02:33.658 INFO:teuthology.orchestra.run.smithi053.stdout:2115 write 0xc1b8d2b thru 0xc1bb4fc (0x27d2 bytes) 2024-09-13T05:02:33.661 INFO:teuthology.orchestra.run.smithi053.stdout:2116 punch from 0x3bf4910 to 0x3bfb067, (0x6757 bytes) 2024-09-13T05:02:33.661 INFO:teuthology.orchestra.run.smithi053.stdout:2117 writesame 0x56476f8 thru 0x564e7bb data_size 0xe(0x70c4 bytes) 2024-09-13T05:02:33.664 INFO:teuthology.orchestra.run.smithi053.stdout:2119 punch from 0xaba8ee1 to 0xabb1327, (0x8446 bytes) 2024-09-13T05:02:33.664 INFO:teuthology.orchestra.run.smithi053.stdout:2120 read 0x819dab6 thru 0x81aba69 (0xdfb4 bytes) 2024-09-13T05:02:33.664 INFO:teuthology.orchestra.run.smithi053.stdout:2121 write 0x2ca5b79 thru 0x2cb2cda (0xd162 bytes) 2024-09-13T05:02:33.668 INFO:teuthology.orchestra.run.smithi053.stdout:2124 read 0x5ae82c7 thru 0x5af7a3e (0xf778 bytes) 2024-09-13T05:02:33.668 INFO:teuthology.orchestra.run.smithi053.stdout:2125 writesame 0x9428005 thru 0x9429ec6 data_size 0x1f(0x1ec2 bytes) 2024-09-13T05:02:33.670 INFO:teuthology.orchestra.run.smithi053.stdout:2126 write 0x65c22eb thru 0x65d1d6a (0xfa80 bytes) 2024-09-13T05:02:33.674 INFO:teuthology.orchestra.run.smithi053.stdout:2128 read 0x18a0d7 thru 0x198b46 (0xea70 bytes) 2024-09-13T05:02:33.674 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.677 INFO:teuthology.orchestra.run.smithi053.stdout:2130 trunc from 0xc1bb4fd to 0xa0534bb 2024-09-13T05:02:33.694 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:33.694+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:33.698 INFO:teuthology.orchestra.run.smithi053.stdout:2131 trunc from 0xa0534bb to 0x555668 2024-09-13T05:02:33.763 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:33.763 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:33.768 INFO:teuthology.orchestra.run.smithi053.stdout:2132 writesame 0xe7a16e4 thru 0xe7a6b14 data_size 0x1(0x5431 bytes) 2024-09-13T05:02:33.768 INFO:teuthology.orchestra.run.smithi053.stdout:2133 read 0x69ce3ed thru 0x69d6015 (0x7c29 bytes) 2024-09-13T05:02:33.769 INFO:teuthology.orchestra.run.smithi053.stdout:2135 read 0x764ac7e thru 0x7650435 (0x57b8 bytes) 2024-09-13T05:02:33.769 INFO:teuthology.orchestra.run.smithi053.stdout:2136 write 0x9c310f3 thru 0x9c3287c (0x178a bytes) 2024-09-13T05:02:33.771 INFO:teuthology.orchestra.run.smithi053.stdout:2137 read 0x8b059d0 thru 0x8b0d042 (0x7673 bytes) 2024-09-13T05:02:33.771 INFO:teuthology.orchestra.run.smithi053.stdout:2138 read 0xe03c84c thru 0xe040c35 (0x43ea bytes) 2024-09-13T05:02:33.772 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:33.785 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:33.784+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:33.787 INFO:teuthology.orchestra.run.smithi053.stdout:2139 trunc from 0xe7a6b15 to 0x348f77 2024-09-13T05:02:33.809 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.809 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.809 INFO:teuthology.orchestra.run.smithi053.stdout:2142 read 0x23e6d3 thru 0x24807f (0x99ad bytes) 2024-09-13T05:02:33.809 INFO:teuthology.orchestra.run.smithi053.stdout:2144 read 0x1d3c3b thru 0x1e276f (0xeb35 bytes) 2024-09-13T05:02:33.815 INFO:teuthology.orchestra.run.smithi053.stdout:2145 writesame 0x3c39e01 thru 0x3c40d9e data_size 0x1a(0x6f9e bytes) 2024-09-13T05:02:33.826 INFO:teuthology.orchestra.run.smithi053.stdout:2146 write 0x8da3db3 thru 0x8daef6a (0xb1b8 bytes) 2024-09-13T05:02:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:2148 read 0x27bd8e9 thru 0x27bf702 (0x1e1a bytes) 2024-09-13T05:02:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.829 INFO:teuthology.orchestra.run.smithi053.stdout:2150 writesame 0x7af0d8c thru 0x7af1377 data_size 0x2(0x5ec bytes) 2024-09-13T05:02:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:2151 read 0x7867127 thru 0x7867bb4 (0xa8e bytes) 2024-09-13T05:02:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:2152 punch from 0x7559251 to 0x755e9d8, (0x5787 bytes) 2024-09-13T05:02:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:2156 read 0x272045e thru 0x2727aa7 (0x764a bytes) 2024-09-13T05:02:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.832 INFO:teuthology.orchestra.run.smithi053.stdout:2158 writesame 0x80c0545 thru 0x80cfbcd data_size 0x1(0xf689 bytes) 2024-09-13T05:02:33.861 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.861 INFO:teuthology.orchestra.run.smithi053.stdout:2161 punch from 0x3c7806c to 0x3c7a586, (0x251a bytes) 2024-09-13T05:02:33.861 INFO:teuthology.orchestra.run.smithi053.stdout:2162 write 0x72aae6f thru 0x72b2e43 (0x7fd5 bytes) 2024-09-13T05:02:33.865 INFO:teuthology.orchestra.run.smithi053.stdout:2163 writesame 0x30c90b7 thru 0x30d008b data_size 0x3(0x6fd5 bytes) 2024-09-13T05:02:33.871 INFO:teuthology.orchestra.run.smithi053.stdout:2164 writesame 0x62d22ec thru 0x62d73bd data_size 0x5(0x50d2 bytes) 2024-09-13T05:02:33.875 INFO:teuthology.orchestra.run.smithi053.stdout:2165 read 0x2a8ce2 thru 0x2b68bf (0xdbde bytes) 2024-09-13T05:02:33.880 INFO:teuthology.orchestra.run.smithi053.stdout:2166 trunc from 0x8daef6b to 0x550a02d 2024-09-13T05:02:33.886 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:33.885+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:33.889 INFO:teuthology.orchestra.run.smithi053.stdout:2167 punch from 0x80191f to 0x8063d8, (0x4ab9 bytes) 2024-09-13T05:02:33.899 INFO:teuthology.orchestra.run.smithi053.stdout:2171 write 0x91e7548 thru 0x91f7382 (0xfe3b bytes) 2024-09-13T05:02:33.902 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.902 INFO:teuthology.orchestra.run.smithi053.stdout:2176 punch from 0x4e01149 to 0x4e056c1, (0x4578 bytes) 2024-09-13T05:02:33.902 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.902 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.911 INFO:teuthology.orchestra.run.smithi053.stdout:2180 write 0xc50cdb6 thru 0xc51a1d0 (0xd41b bytes) 2024-09-13T05:02:33.917 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:33.918 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:33.918 INFO:teuthology.orchestra.run.smithi053.stdout:2181 trunc from 0xc51a1d1 to 0x905dbee 2024-09-13T05:02:33.926 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:33.933 INFO:teuthology.orchestra.run.smithi053.stdout:2182 punch from 0xb030bc to 0xb12e59, (0xfd9d bytes) 2024-09-13T05:02:33.933 INFO:teuthology.orchestra.run.smithi053.stdout:2183 read 0xd3f15b thru 0xd4c8df (0xd785 bytes) 2024-09-13T05:02:33.933 INFO:teuthology.orchestra.run.smithi053.stdout:2184 read 0x4f62232 thru 0x4f711e1 (0xefb0 bytes) 2024-09-13T05:02:33.945 INFO:teuthology.orchestra.run.smithi053.stdout:2188 writesame 0xee018b7 thru 0xee02ae8 data_size 0x11(0x1232 bytes) 2024-09-13T05:02:33.945 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.945 INFO:teuthology.orchestra.run.smithi053.stdout:2190 read 0xb75fe99 thru 0xb76d6c3 (0xd82b bytes) 2024-09-13T05:02:33.945 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.945 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:33.945 INFO:teuthology.orchestra.run.smithi053.stdout:2193 read 0x4b69c9b thru 0x4b73d2b (0xa091 bytes) 2024-09-13T05:02:33.946 INFO:teuthology.orchestra.run.smithi053.stdout:2194 read 0x983a5eb thru 0x9842779 (0x818f bytes) 2024-09-13T05:02:33.960 INFO:teuthology.orchestra.run.smithi053.stdout:2195 trunc from 0xee02ae9 to 0x1adc710 2024-09-13T05:02:33.978 INFO:teuthology.orchestra.run.smithi053.stdout:2197 write 0x4e2cf46 thru 0x4e39ecc (0xcf87 bytes) 2024-09-13T05:02:33.983 INFO:teuthology.orchestra.run.smithi053.stdout:2198 read 0x2b8dd04 thru 0x2b9347f (0x577c bytes) 2024-09-13T05:02:33.987 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:33.986+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:33.992 INFO:teuthology.orchestra.run.smithi053.stdout:2199 write 0x8975dbb thru 0x897d9e6 (0x7c2c bytes) 2024-09-13T05:02:33.994 INFO:teuthology.orchestra.run.smithi053.stdout:2201 read 0x490151a thru 0x490783b (0x6322 bytes) 2024-09-13T05:02:34.002 INFO:teuthology.orchestra.run.smithi053.stdout:2202 write 0xa3c15b6 thru 0xa3cb25f (0x9caa bytes) 2024-09-13T05:02:34.005 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.015 INFO:teuthology.orchestra.run.smithi053.stdout:2205 writesame 0xe715cbf thru 0xe71b763 data_size 0x5b(0x5aa5 bytes) 2024-09-13T05:02:34.015 INFO:teuthology.orchestra.run.smithi053.stdout:2208 read 0xc61495a thru 0xc614a48 (0xef bytes) 2024-09-13T05:02:34.016 INFO:teuthology.orchestra.run.smithi053.stdout:2210 trunc from 0xe71b764 to 0xd256ff8 2024-09-13T05:02:34.021 INFO:teuthology.orchestra.run.smithi053.stdout:2211 writesame 0xb20bdc6 thru 0xb21b447 data_size 0x8b(0xf682 bytes) 2024-09-13T05:02:34.025 INFO:teuthology.orchestra.run.smithi053.stdout:2212 punch from 0x7be6a7e to 0x7be809c, (0x161e bytes) 2024-09-13T05:02:34.026 INFO:teuthology.orchestra.run.smithi053.stdout:2213 trunc from 0xd256ff8 to 0xc5385d5 2024-09-13T05:02:34.032 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.032 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.033 INFO:teuthology.orchestra.run.smithi053.stdout:2214 trunc from 0xc5385d5 to 0xe25aedd 2024-09-13T05:02:34.041 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:34.049 INFO:teuthology.orchestra.run.smithi053.stdout:2215 trunc from 0xe25aedd to 0x41ea0bb 2024-09-13T05:02:34.064 INFO:teuthology.orchestra.run.smithi053.stdout:2216 trunc from 0x41ea0bb to 0x85bdcb3 2024-09-13T05:02:34.075 INFO:teuthology.orchestra.run.smithi053.stdout:2217 trunc from 0x85bdcb3 to 0x320e5fc 2024-09-13T05:02:34.080 INFO:teuthology.orchestra.run.smithi053.stdout:2218 read 0xb0c984 thru 0xb14579 (0x7bf6 bytes) 2024-09-13T05:02:34.087 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:34.087+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:34.094 INFO:teuthology.orchestra.run.smithi053.stdout:2219 write 0xad3b099 thru 0xad47791 (0xc6f9 bytes) 2024-09-13T05:02:34.099 INFO:teuthology.orchestra.run.smithi053.stdout:2220 write 0xa91f227 thru 0xa92a579 (0xb353 bytes) 2024-09-13T05:02:34.102 INFO:teuthology.orchestra.run.smithi053.stdout:2221 write 0x59316be thru 0x5935ea4 (0x47e7 bytes) 2024-09-13T05:02:34.105 INFO:teuthology.orchestra.run.smithi053.stdout:2223 write 0x615b265 thru 0x6166a2a (0xb7c6 bytes) 2024-09-13T05:02:34.109 INFO:teuthology.orchestra.run.smithi053.stdout:2224 punch from 0x60427e8 to 0x604e524, (0xbd3c bytes) 2024-09-13T05:02:34.109 INFO:teuthology.orchestra.run.smithi053.stdout:2225 read 0xa5e8575 thru 0xa5ef7aa (0x7236 bytes) 2024-09-13T05:02:34.115 INFO:teuthology.orchestra.run.smithi053.stdout:2226 write 0xb6960d7 thru 0xb6a15af (0xb4d9 bytes) 2024-09-13T05:02:34.119 INFO:teuthology.orchestra.run.smithi053.stdout:2232 punch from 0x79123af to 0x7912c13, (0x864 bytes) 2024-09-13T05:02:34.119 INFO:teuthology.orchestra.run.smithi053.stdout:2233 write 0x26c5a03 thru 0x26c9598 (0x3b96 bytes) 2024-09-13T05:02:34.130 INFO:teuthology.orchestra.run.smithi053.stdout:2234 write 0xdd2947c thru 0xdd390e7 (0xfc6c bytes) 2024-09-13T05:02:34.140 INFO:teuthology.orchestra.run.smithi053.stdout:2235 trunc from 0xdd390e8 to 0x7dcf55a 2024-09-13T05:02:34.147 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.147 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.147 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:34.148 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:34.148 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T05:02:34.148 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-09-13T05:02:34.148 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T05:02:34.151 INFO:teuthology.orchestra.run.smithi053.stdout:2236 read 0x23b3252 thru 0x23b6d1d (0x3acc bytes) 2024-09-13T05:02:34.151 INFO:teuthology.orchestra.run.smithi053.stdout:2237 write 0x41803ab thru 0x41867e6 (0x643c bytes) 2024-09-13T05:02:34.153 INFO:teuthology.orchestra.run.smithi053.stdout:2239 write 0x56d29b9 thru 0x56d8633 (0x5c7b bytes) 2024-09-13T05:02:34.156 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:34.156 INFO:teuthology.orchestra.run.smithi053.stdout:2241 read 0x79bc311 thru 0x79c9f20 (0xdc10 bytes) 2024-09-13T05:02:34.166 INFO:teuthology.orchestra.run.smithi053.stdout:2242 writesame 0xc01505c thru 0xc0246dc data_size 0x15(0xf681 bytes) 2024-09-13T05:02:34.167 INFO:teuthology.orchestra.run.smithi053.stdout:2243 writesame 0x8637818 thru 0x863bfab data_size 0x12(0x4794 bytes) 2024-09-13T05:02:34.169 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.169 INFO:teuthology.orchestra.run.smithi053.stdout:2245 read 0x23a3685 thru 0x23a8bb9 (0x5535 bytes) 2024-09-13T05:02:34.170 INFO:teuthology.orchestra.run.smithi053.stdout:2246 write 0x6157fe4 thru 0x615f3d4 (0x73f1 bytes) 2024-09-13T05:02:34.171 INFO:teuthology.orchestra.run.smithi053.stdout:2247 write 0xb3cb726 thru 0xb3d2c5a (0x7535 bytes) 2024-09-13T05:02:34.180 INFO:teuthology.orchestra.run.smithi053.stdout:2248 write 0xc1c592e thru 0xc1cf105 (0x97d8 bytes) 2024-09-13T05:02:34.184 INFO:teuthology.orchestra.run.smithi053.stdout:2249 punch from 0x2da709e to 0x2db1325, (0xa287 bytes) 2024-09-13T05:02:34.184 INFO:teuthology.orchestra.run.smithi053.stdout:2251 read 0xa6c66cc thru 0xa6d637f (0xfcb4 bytes) 2024-09-13T05:02:34.191 INFO:teuthology.orchestra.run.smithi053.stdout:2253 write 0xdd88c73 thru 0xdd968f9 (0xdc87 bytes) 2024-09-13T05:02:34.195 INFO:teuthology.orchestra.run.smithi053.stdout:2254 write 0xb94f071 thru 0xb94f121 (0xb1 bytes) 2024-09-13T05:02:34.197 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:34.197+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:34.198 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.198 INFO:teuthology.orchestra.run.smithi053.stdout:2257 read 0xc845d0b thru 0xc847c89 (0x1f7f bytes) 2024-09-13T05:02:34.198 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.208 INFO:teuthology.orchestra.run.smithi053.stdout:2260 trunc from 0xdd968fa to 0x495123b 2024-09-13T05:02:34.219 INFO:teuthology.orchestra.run.smithi053.stdout:2261 read 0x36c0a19 thru 0x36c3561 (0x2b49 bytes) 2024-09-13T05:02:34.234 INFO:teuthology.orchestra.run.smithi053.stdout:2262 write 0xd1c9e9d thru 0xd1cfa2c (0x5b90 bytes) 2024-09-13T05:02:34.238 INFO:teuthology.orchestra.run.smithi053.stdout:2264 write 0x4512902 thru 0x451cdda (0xa4d9 bytes) 2024-09-13T05:02:34.241 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.241 INFO:teuthology.orchestra.run.smithi053.stdout:2266 read 0x20590ae thru 0x205949b (0x3ee bytes) 2024-09-13T05:02:34.241 INFO:teuthology.orchestra.run.smithi053.stdout:2270 punch from 0xc542e6d to 0xc54de2b, (0xafbe bytes) 2024-09-13T05:02:34.241 INFO:teuthology.orchestra.run.smithi053.stdout:2273 writesame 0x5794007 thru 0x57a0742 data_size 0x52(0xc73c bytes) 2024-09-13T05:02:34.250 INFO:teuthology.orchestra.run.smithi053.stdout:2274 writesame 0xd2db984 thru 0xd2e2a9e data_size 0x1(0x711b bytes) 2024-09-13T05:02:34.250 INFO:teuthology.orchestra.run.smithi053.stdout:2277 punch from 0x7d8841b to 0x7d8c21a, (0x3dff bytes) 2024-09-13T05:02:34.251 INFO:teuthology.orchestra.run.smithi053.stdout:2281 trunc from 0xd2e2a9f to 0xbec8b69 2024-09-13T05:02:34.268 INFO:teuthology.orchestra.run.smithi053.stdout:2282 trunc from 0xbec8b69 to 0x49ef369 2024-09-13T05:02:34.279 INFO:teuthology.orchestra.run.smithi053.stdout:2283 write 0x22fd792 thru 0x2300c18 (0x3487 bytes) 2024-09-13T05:02:34.289 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:34.289+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:34.297 INFO:teuthology.orchestra.run.smithi053.stdout:2284 write 0xc92527e thru 0xc92ff4c (0xaccf bytes) 2024-09-13T05:02:34.300 INFO:teuthology.orchestra.run.smithi053.stdout:2285 read 0x1cad31f thru 0x1cb2192 (0x4e74 bytes) 2024-09-13T05:02:34.304 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.304 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.307 INFO:teuthology.orchestra.run.smithi053.stdout:2286 write 0xe8ce95a thru 0xe8d9d55 (0xb3fc bytes) 2024-09-13T05:02:34.311 INFO:teuthology.orchestra.run.smithi053.stdout:2287 write 0xc2efb1b thru 0xc2f3353 (0x3839 bytes) 2024-09-13T05:02:34.314 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:34.315 INFO:teuthology.orchestra.run.smithi053.stdout:2288 read 0xb062b6e thru 0xb062c64 (0xf7 bytes) 2024-09-13T05:02:34.315 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.315 INFO:teuthology.orchestra.run.smithi053.stdout:2290 read 0x565f736 thru 0x566674b (0x7016 bytes) 2024-09-13T05:02:34.315 INFO:teuthology.orchestra.run.smithi053.stdout:2291 writesame 0xc4950f6 thru 0xc49f68b data_size 0x5a(0xa596 bytes) 2024-09-13T05:02:34.317 INFO:teuthology.orchestra.run.smithi053.stdout:2292 read 0x8a54ced thru 0x8a63ae3 (0xedf7 bytes) 2024-09-13T05:02:34.318 INFO:teuthology.orchestra.run.smithi053.stdout:2296 write 0xdfbd409 thru 0xdfca5d9 (0xd1d1 bytes) 2024-09-13T05:02:34.322 INFO:teuthology.orchestra.run.smithi053.stdout:2297 punch from 0x87efc25 to 0x87f6256, (0x6631 bytes) 2024-09-13T05:02:34.322 INFO:teuthology.orchestra.run.smithi053.stdout:2298 read 0x2ba45a thru 0x2c4bf9 (0xa7a0 bytes) 2024-09-13T05:02:34.322 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.323 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.323 INFO:teuthology.orchestra.run.smithi053.stdout:2302 write 0xdc06ef9 thru 0xdc1615f (0xf267 bytes) 2024-09-13T05:02:34.326 INFO:teuthology.orchestra.run.smithi053.stdout:2303 punch from 0xbcefa to 0xc72a4, (0xa3aa bytes) 2024-09-13T05:02:34.326 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.327 INFO:teuthology.orchestra.run.smithi053.stdout:2306 write 0x90ac66f thru 0x90b1db1 (0x5743 bytes) 2024-09-13T05:02:34.330 INFO:teuthology.orchestra.run.smithi053.stdout:2307 punch from 0xe0caa45 to 0xe0d25a4, (0x7b5f bytes) 2024-09-13T05:02:34.330 INFO:teuthology.orchestra.run.smithi053.stdout:2308 read 0x557d6ee thru 0x5587af9 (0xa40c bytes) 2024-09-13T05:02:34.330 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.346 INFO:teuthology.orchestra.run.smithi053.stdout:2311 trunc from 0xe8d9d56 to 0x34f7b6c 2024-09-13T05:02:34.358 INFO:teuthology.orchestra.run.smithi053.stdout:2312 punch from 0x2449e8a to 0x2452fa7, (0x911d bytes) 2024-09-13T05:02:34.358 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.369 INFO:teuthology.orchestra.run.smithi053.stdout:2314 trunc from 0x34f7b6c to 0xb4c29d3 2024-09-13T05:02:34.375 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.380 INFO:teuthology.orchestra.run.smithi053.stdout:2317 trunc from 0xb4c29d3 to 0x6ede082 2024-09-13T05:02:34.391 INFO:teuthology.orchestra.run.smithi053.stdout:2318 trunc from 0x6ede082 to 0xc91b0bc 2024-09-13T05:02:34.396 INFO:teuthology.orchestra.run.smithi053.stdout:2319 read 0xb03691f thru 0xb043fd8 (0xd6ba bytes) 2024-09-13T05:02:34.397 INFO:teuthology.orchestra.run.smithi053.stdout:2321 read 0x754b0df thru 0x75538fc (0x881e bytes) 2024-09-13T05:02:34.397 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:34.397+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:34.402 INFO:teuthology.orchestra.run.smithi053.stdout:2322 writesame 0xcdb8559 thru 0xcdbde2a data_size 0x1(0x58d2 bytes) 2024-09-13T05:02:34.402 INFO:teuthology.orchestra.run.smithi053.stdout:2324 write 0x8baeab4 thru 0x8bbb68d (0xcbda bytes) 2024-09-13T05:02:34.406 INFO:teuthology.orchestra.run.smithi053.stdout:2325 write 0x77a6160 thru 0x77ad0a2 (0x6f43 bytes) 2024-09-13T05:02:34.409 INFO:teuthology.orchestra.run.smithi053.stdout:2326 write 0x4d875f4 thru 0x4d8c485 (0x4e92 bytes) 2024-09-13T05:02:34.413 INFO:teuthology.orchestra.run.smithi053.stdout:2327 punch from 0x84b6757 to 0x84ba553, (0x3dfc bytes) 2024-09-13T05:02:34.413 INFO:teuthology.orchestra.run.smithi053.stdout:2330 punch from 0x412cdba to 0x413c750, (0xf996 bytes) 2024-09-13T05:02:34.413 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.413 INFO:teuthology.orchestra.run.smithi053.stdout:2332 punch from 0x8ba6e5d to 0x8badc35, (0x6dd8 bytes) 2024-09-13T05:02:34.419 INFO:teuthology.orchestra.run.smithi053.stdout:2334 writesame 0xdb4f94f thru 0xdb5ca0f data_size 0x1(0xd0c1 bytes) 2024-09-13T05:02:34.419 INFO:teuthology.orchestra.run.smithi053.stdout:2335 read 0x6318a67 thru 0x631b1ff (0x2799 bytes) 2024-09-13T05:02:34.426 INFO:teuthology.orchestra.run.smithi053.stdout:2337 write 0xe9ccef2 thru 0xe9d84d4 (0xb5e3 bytes) 2024-09-13T05:02:34.429 INFO:teuthology.orchestra.run.smithi053.stdout:2338 write 0x84942e4 thru 0x849ff03 (0xbc20 bytes) 2024-09-13T05:02:34.432 INFO:teuthology.orchestra.run.smithi053.stdout:2339 read 0x7fcf366 thru 0x7fcf84e (0x4e9 bytes) 2024-09-13T05:02:34.432 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.433 INFO:teuthology.orchestra.run.smithi053.stdout:2341 write 0x46393d6 thru 0x463951e (0x149 bytes) 2024-09-13T05:02:34.449 INFO:teuthology.orchestra.run.smithi053.stdout:2342 trunc from 0xe9d84d5 to 0x31322db 2024-09-13T05:02:34.460 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.460 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.460 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.460 INFO:teuthology.orchestra.run.smithi053.stdout:2344 flatten 2024-09-13T05:02:34.466 INFO:teuthology.orchestra.run.smithi053.stdout:2345 punch from 0x2b868c6 to 0x2b8ebd7, (0x8311 bytes) 2024-09-13T05:02:34.466 INFO:teuthology.orchestra.run.smithi053.stdout:2346 read 0x259f3a0 thru 0x25a8fa7 (0x9c08 bytes) 2024-09-13T05:02:34.469 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:34.475 INFO:teuthology.orchestra.run.smithi053.stdout:2347 write 0x4fd49dd thru 0x4fe065d (0xbc81 bytes) 2024-09-13T05:02:34.494 INFO:teuthology.orchestra.run.smithi053.stdout:2348 writesame 0xc98d964 thru 0xc992dc7 data_size 0x16(0x5464 bytes) 2024-09-13T05:02:34.494 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.494 INFO:teuthology.orchestra.run.smithi053.stdout:2352 read 0xa032a77 thru 0xa042114 (0xf69e bytes) 2024-09-13T05:02:34.498 INFO:teuthology.orchestra.run.smithi053.stdout:2353 trunc from 0xc992dc8 to 0x90b9709 2024-09-13T05:02:34.504 INFO:teuthology.orchestra.run.smithi053.stdout:2354 punch from 0x200697c to 0x20139d0, (0xd054 bytes) 2024-09-13T05:02:34.505 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.505 INFO:teuthology.orchestra.run.smithi053.stdout:2356 punch from 0x6d59617 to 0x6d5fd40, (0x6729 bytes) 2024-09-13T05:02:34.505 INFO:teuthology.orchestra.run.smithi053.stdout:2357 writesame 0x3b706c5 thru 0x3b78628 data_size 0x1f(0x7f64 bytes) 2024-09-13T05:02:34.507 INFO:teuthology.orchestra.run.smithi053.stdout:2358 writesame 0x7d85f1c thru 0x7d8dda7 data_size 0x5b(0x7e8c bytes) 2024-09-13T05:02:34.509 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:34.509+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:34.510 INFO:teuthology.orchestra.run.smithi053.stdout:2360 write 0x7a0cfa9 thru 0x7a13a52 (0x6aaa bytes) 2024-09-13T05:02:34.517 INFO:teuthology.orchestra.run.smithi053.stdout:2361 trunc from 0x90b9709 to 0xae44e20 2024-09-13T05:02:34.523 INFO:teuthology.orchestra.run.smithi053.stdout:2362 read 0x71ed860 thru 0x71eea60 (0x1201 bytes) 2024-09-13T05:02:34.523 INFO:teuthology.orchestra.run.smithi053.stdout:2363 punch from 0x96249b5 to 0x96319d5, (0xd020 bytes) 2024-09-13T05:02:34.531 INFO:teuthology.orchestra.run.smithi053.stdout:2364 writesame 0xcef2584 thru 0xcefab7e data_size 0x25(0x85fb bytes) 2024-09-13T05:02:34.538 INFO:teuthology.orchestra.run.smithi053.stdout:2365 write 0xe4648a8 thru 0xe46fb73 (0xb2cc bytes) 2024-09-13T05:02:34.541 INFO:teuthology.orchestra.run.smithi053.stdout:2366 read 0x651c66a thru 0x6527733 (0xb0ca bytes) 2024-09-13T05:02:34.541 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.541 INFO:teuthology.orchestra.run.smithi053.stdout:2369 trunc from 0xe46fb74 to 0xe4d343a 2024-09-13T05:02:34.545 INFO:teuthology.orchestra.run.smithi053.stdout:2370 write 0x9a2e79b thru 0x9a31dff (0x3665 bytes) 2024-09-13T05:02:34.549 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.549 INFO:teuthology.orchestra.run.smithi053.stdout:2374 read 0xa32a72f thru 0xa33a628 (0xfefa bytes) 2024-09-13T05:02:34.549 INFO:teuthology.orchestra.run.smithi053.stdout:2375 write 0x2fba644 thru 0x2fc9b88 (0xf545 bytes) 2024-09-13T05:02:34.554 INFO:teuthology.orchestra.run.smithi053.stdout:2377 read 0x77ee2fb thru 0x77f56d7 (0x73dd bytes) 2024-09-13T05:02:34.554 INFO:teuthology.orchestra.run.smithi053.stdout:2378 write 0xcd83934 thru 0xcd879dc (0x40a9 bytes) 2024-09-13T05:02:34.557 INFO:teuthology.orchestra.run.smithi053.stdout:2379 read 0x68fbd85 thru 0x6901e7c (0x60f8 bytes) 2024-09-13T05:02:34.557 INFO:teuthology.orchestra.run.smithi053.stdout:2381 read 0x2ae392d thru 0x2ae3fc3 (0x697 bytes) 2024-09-13T05:02:34.557 INFO:teuthology.orchestra.run.smithi053.stdout:2382 write 0x436ab98 thru 0x436f0a5 (0x450e bytes) 2024-09-13T05:02:34.560 INFO:teuthology.orchestra.run.smithi053.stdout:2383 read 0x74e6d3a thru 0x74edcf2 (0x6fb9 bytes) 2024-09-13T05:02:34.560 INFO:teuthology.orchestra.run.smithi053.stdout:2384 write 0x420b8aa thru 0x42152aa (0x9a01 bytes) 2024-09-13T05:02:34.563 INFO:teuthology.orchestra.run.smithi053.stdout:2386 write 0x85398d8 thru 0x8540dca (0x74f3 bytes) 2024-09-13T05:02:34.567 INFO:teuthology.orchestra.run.smithi053.stdout:2387 read 0xc23b860 thru 0xc23bb7c (0x31d bytes) 2024-09-13T05:02:34.567 INFO:teuthology.orchestra.run.smithi053.stdout:2388 punch from 0xbec2160 to 0xbece806, (0xc6a6 bytes) 2024-09-13T05:02:34.567 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.567 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.572 INFO:teuthology.orchestra.run.smithi053.stdout:2393 write 0xea59a32 thru 0xea607a7 (0x6d76 bytes) 2024-09-13T05:02:34.575 INFO:teuthology.orchestra.run.smithi053.stdout:2395 read 0x5393624 thru 0x53939f9 (0x3d6 bytes) 2024-09-13T05:02:34.575 INFO:teuthology.orchestra.run.smithi053.stdout:2396 writesame 0x7cc9a21 thru 0x7cd52b7 data_size 0xd(0xb897 bytes) 2024-09-13T05:02:34.579 INFO:teuthology.orchestra.run.smithi053.stdout:2397 write 0xbe3de97 thru 0xbe3e7a8 (0x912 bytes) 2024-09-13T05:02:34.582 INFO:teuthology.orchestra.run.smithi053.stdout:2399 writesame 0x64d8f90 thru 0x64defa2 data_size 0x1(0x6013 bytes) 2024-09-13T05:02:34.592 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:34.592+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:34.594 INFO:teuthology.orchestra.run.smithi053.stdout:2400 write 0x1d74320 thru 0x1d7ad2d (0x6a0e bytes) 2024-09-13T05:02:34.598 INFO:teuthology.orchestra.run.smithi053.stdout:2402 write 0x97936f0 thru 0x979adf8 (0x7709 bytes) 2024-09-13T05:02:34.601 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.601 INFO:teuthology.orchestra.run.smithi053.stdout:2405 punch from 0x1844437 to 0x18464a1, (0x206a bytes) 2024-09-13T05:02:34.613 INFO:teuthology.orchestra.run.smithi053.stdout:2407 trunc from 0xea607a8 to 0x3be0d8e 2024-09-13T05:02:34.614 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.614 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.625 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:34.626 INFO:teuthology.orchestra.run.smithi053.stdout:2410 read 0x72a60d thru 0x72e896 (0x428a bytes) 2024-09-13T05:02:34.641 INFO:teuthology.orchestra.run.smithi053.stdout:2411 write 0xcb2b4f7 thru 0xcb30417 (0x4f21 bytes) 2024-09-13T05:02:34.645 INFO:teuthology.orchestra.run.smithi053.stdout:2412 write 0x6c1ff80 thru 0x6c257c2 (0x5843 bytes) 2024-09-13T05:02:34.649 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.649 INFO:teuthology.orchestra.run.smithi053.stdout:2414 write 0x619fe09 thru 0x61a7b5f (0x7d57 bytes) 2024-09-13T05:02:34.658 INFO:teuthology.orchestra.run.smithi053.stdout:2415 write 0xd1e8740 thru 0xd1f3b80 (0xb441 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2416 read 0x92b1b6f thru 0x92be3ae (0xc840 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2417 read 0x82425c1 thru 0x8243125 (0xb65 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2419 punch from 0x75a0ced to 0x75a9006, (0x8319 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2420 punch from 0x9a11303 to 0x9a17666, (0x6363 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2421 read 0x267f7e7 thru 0x268ac33 (0xb44d bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2422 read 0x1246580 thru 0x124c813 (0x6294 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2426 punch from 0x9db2960 to 0x9dc1bb1, (0xf251 bytes) 2024-09-13T05:02:34.663 INFO:teuthology.orchestra.run.smithi053.stdout:2427 punch from 0x952e788 to 0x9537926, (0x919e bytes) 2024-09-13T05:02:34.674 INFO:teuthology.orchestra.run.smithi053.stdout:2428 trunc from 0xd1f3b81 to 0x266813d 2024-09-13T05:02:34.694 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:34.693+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:34.706 INFO:teuthology.orchestra.run.smithi053.stdout:2429 write 0xe53a7da thru 0xe5471a3 (0xc9ca bytes) 2024-09-13T05:02:34.709 INFO:teuthology.orchestra.run.smithi053.stdout:2430 write 0x28d6a82 thru 0x28d6e7e (0x3fd bytes) 2024-09-13T05:02:34.723 INFO:teuthology.orchestra.run.smithi053.stdout:2431 trunc from 0xe5471a4 to 0x5308420 2024-09-13T05:02:34.732 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.732 INFO:teuthology.orchestra.run.smithi053.stdout:2433 writesame 0x2a437df thru 0x2a4fcb7 data_size 0x17(0xc4d9 bytes) 2024-09-13T05:02:34.736 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.736 INFO:teuthology.orchestra.run.smithi053.stdout:2435 write 0x2b15a65 thru 0x2b17551 (0x1aed bytes) 2024-09-13T05:02:34.739 INFO:teuthology.orchestra.run.smithi053.stdout:2438 read 0x10a2e84 thru 0x10a3e38 (0xfb5 bytes) 2024-09-13T05:02:34.744 INFO:teuthology.orchestra.run.smithi053.stdout:2439 trunc from 0x5308420 to 0x974d54 2024-09-13T05:02:34.764 INFO:teuthology.orchestra.run.smithi053.stdout:2440 trunc from 0x974d54 to 0x9f62b6b 2024-09-13T05:02:34.771 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.771 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.776 INFO:teuthology.orchestra.run.smithi053.stdout:2441 write 0xb0491cb thru 0xb051c8f (0x8ac5 bytes) 2024-09-13T05:02:34.779 INFO:teuthology.orchestra.run.smithi053.stdout:2442 write 0xed3159 thru 0xed7383 (0x422b bytes) 2024-09-13T05:02:34.781 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:34.782 INFO:teuthology.orchestra.run.smithi053.stdout:2443 write 0x6dd4587 thru 0x6ddb8eb (0x7365 bytes) 2024-09-13T05:02:34.785 INFO:teuthology.orchestra.run.smithi053.stdout:2444 punch from 0x49cbae4 to 0x49d547b, (0x9997 bytes) 2024-09-13T05:02:34.785 INFO:teuthology.orchestra.run.smithi053.stdout:2445 punch from 0xa1d21cf to 0xa1debd4, (0xca05 bytes) 2024-09-13T05:02:34.785 INFO:teuthology.orchestra.run.smithi053.stdout:2446 write 0x2f475a1 thru 0x2f4d7a5 (0x6205 bytes) 2024-09-13T05:02:34.788 INFO:teuthology.orchestra.run.smithi053.stdout:2447 write 0xfefb7c thru 0xff59c7 (0x5e4c bytes) 2024-09-13T05:02:34.801 INFO:teuthology.orchestra.run.smithi053.stdout:2448 write 0xe97b9ae thru 0xe98972c (0xdd7f bytes) 2024-09-13T05:02:34.820 INFO:teuthology.orchestra.run.smithi053.stdout:2449 trunc from 0xe98972d to 0x71fe70 2024-09-13T05:02:34.825 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:34.824+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:34.843 INFO:teuthology.orchestra.run.smithi053.stdout:2450 writesame 0x7ff7326 thru 0x7ffa025 data_size 0x3c(0x2d00 bytes) 2024-09-13T05:02:34.843 INFO:teuthology.orchestra.run.smithi053.stdout:2453 write 0x6f567bf thru 0x6f5e5b3 (0x7df5 bytes) 2024-09-13T05:02:34.849 INFO:teuthology.orchestra.run.smithi053.stdout:2454 trunc from 0x7ffa026 to 0xaa86bdb 2024-09-13T05:02:34.864 INFO:teuthology.orchestra.run.smithi053.stdout:2455 trunc from 0xaa86bdb to 0x19d3149 2024-09-13T05:02:34.886 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:34.886 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:34.886 INFO:teuthology.orchestra.run.smithi053.stdout:2456 write 0x856ca0e thru 0x8578765 (0xbd58 bytes) 2024-09-13T05:02:34.889 INFO:teuthology.orchestra.run.smithi053.stdout:2457 writesame 0x67e0c96 thru 0x67e4cec data_size 0xd(0x4057 bytes) 2024-09-13T05:02:34.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:34.896 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:34.894+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:34.903 INFO:teuthology.orchestra.run.smithi053.stdout:2458 write 0xcc2b3ac thru 0xcc2e76f (0x33c4 bytes) 2024-09-13T05:02:34.906 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:34.906 INFO:teuthology.orchestra.run.smithi053.stdout:2462 write 0x18a88ca thru 0x18b47d9 (0xbf10 bytes) 2024-09-13T05:02:34.909 INFO:teuthology.orchestra.run.smithi053.stdout:2463 read 0x54e823f thru 0x54ed2fb (0x50bd bytes) 2024-09-13T05:02:34.910 INFO:teuthology.orchestra.run.smithi053.stdout:2464 read 0xc867af6 thru 0xc86d0cf (0x55da bytes) 2024-09-13T05:02:34.910 INFO:teuthology.orchestra.run.smithi053.stdout:2466 write 0x166ba34 thru 0x16776d4 (0xbca1 bytes) 2024-09-13T05:02:34.918 INFO:teuthology.orchestra.run.smithi053.stdout:2469 write 0xcfe5636 thru 0xcff2ccb (0xd696 bytes) 2024-09-13T05:02:34.933 INFO:teuthology.orchestra.run.smithi053.stdout:2471 trunc from 0xcff2ccc to 0x2e8e512 2024-09-13T05:02:34.942 INFO:teuthology.orchestra.run.smithi053.stdout:2472 read 0x28f9847 thru 0x28fef7a (0x5734 bytes) 2024-09-13T05:02:34.955 INFO:teuthology.orchestra.run.smithi053.stdout:2473 write 0x9e3c645 thru 0x9e463e1 (0x9d9d bytes) 2024-09-13T05:02:34.970 INFO:teuthology.orchestra.run.smithi053.stdout:2474 trunc from 0x9e463e2 to 0x13d945d 2024-09-13T05:02:34.970 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T05:02:34.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:02:35.002 INFO:teuthology.orchestra.run.smithi053.stdout:2477 writesame 0xa387040 thru 0xa396024 data_size 0x3(0xefe5 bytes) 2024-09-13T05:02:35.002 INFO:teuthology.orchestra.run.smithi053.stdout:2478 writesame 0x171b3dc thru 0x1724e85 data_size 0x3(0x9aaa bytes) 2024-09-13T05:02:35.003 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:35.002+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:35.008 INFO:teuthology.orchestra.run.smithi053.stdout:2480 read 0x1b33fc9 thru 0x1b410a7 (0xd0df bytes) 2024-09-13T05:02:35.008 INFO:teuthology.orchestra.run.smithi053.stdout:2481 read 0x7889b54 thru 0x788efbe (0x546b bytes) 2024-09-13T05:02:35.008 INFO:teuthology.orchestra.run.smithi053.stdout:2482 read 0x961809c thru 0x9624e71 (0xcdd6 bytes) 2024-09-13T05:02:35.008 INFO:teuthology.orchestra.run.smithi053.stdout:2483 write 0x5dfc771 thru 0x5e02469 (0x5cf9 bytes) 2024-09-13T05:02:35.013 INFO:teuthology.orchestra.run.smithi053.stdout:2484 read 0x31e3ff8 thru 0x31f1b25 (0xdb2e bytes) 2024-09-13T05:02:35.013 INFO:teuthology.orchestra.run.smithi053.stdout:2488 read 0x615c2f1 thru 0x6167972 (0xb682 bytes) 2024-09-13T05:02:35.013 INFO:teuthology.orchestra.run.smithi053.stdout:2490 write 0x2372a1b thru 0x23742e1 (0x18c7 bytes) 2024-09-13T05:02:35.016 INFO:teuthology.orchestra.run.smithi053.stdout:2492 write 0x7ee87dd thru 0x7eecddf (0x4603 bytes) 2024-09-13T05:02:35.019 INFO:teuthology.orchestra.run.smithi053.stdout:2493 read 0x2f7caf2 thru 0x2f7de6b (0x137a bytes) 2024-09-13T05:02:35.025 INFO:teuthology.orchestra.run.smithi053.stdout:2494 writesame 0xae5475e thru 0xae5a35f data_size 0x1(0x5c02 bytes) 2024-09-13T05:02:35.026 INFO:teuthology.orchestra.run.smithi053.stdout:2495 read 0x130faee thru 0x131adb6 (0xb2c9 bytes) 2024-09-13T05:02:35.026 INFO:teuthology.orchestra.run.smithi053.stdout:2496 trunc from 0xae5a360 to 0x9cb080d 2024-09-13T05:02:35.032 INFO:teuthology.orchestra.run.smithi053.stdout:2497 writesame 0x718c421 thru 0x7195f21 data_size 0x3(0x9b01 bytes) 2024-09-13T05:02:35.041 INFO:teuthology.orchestra.run.smithi053.stdout:2498 trunc from 0x9cb080d to 0xc732550 2024-09-13T05:02:35.042 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:35.042 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:35.047 INFO:teuthology.orchestra.run.smithi053.stdout:2500 write 0x3d1ddd9 thru 0x3d1de10 (0x38 bytes) 2024-09-13T05:02:35.051 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.052 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:35.053 INFO:teuthology.orchestra.run.smithi053.stdout:2503 trunc from 0xc732550 to 0xe043462 2024-09-13T05:02:35.057 INFO:teuthology.orchestra.run.smithi053.stdout:2505 write 0x52b5ee1 thru 0x52b8a65 (0x2b85 bytes) 2024-09-13T05:02:35.066 INFO:teuthology.orchestra.run.smithi053.stdout:2506 write 0xe52497f thru 0xe53065f (0xbce1 bytes) 2024-09-13T05:02:35.071 INFO:teuthology.orchestra.run.smithi053.stdout:2508 read 0x8aaecd6 thru 0x8ab2371 (0x369c bytes) 2024-09-13T05:02:35.071 INFO:teuthology.orchestra.run.smithi053.stdout:2509 read 0x1baab2e thru 0x1baf050 (0x4523 bytes) 2024-09-13T05:02:35.072 INFO:teuthology.orchestra.run.smithi053.stdout:2510 write 0x2cbfba thru 0x2cf428 (0x346f bytes) 2024-09-13T05:02:35.073 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.073 INFO:teuthology.orchestra.run.smithi053.stdout:2513 read 0xa3d0f thru 0xb252d (0xe81f bytes) 2024-09-13T05:02:35.074 INFO:teuthology.orchestra.run.smithi053.stdout:2515 read 0x2af5cc8 thru 0x2aff819 (0x9b52 bytes) 2024-09-13T05:02:35.074 INFO:teuthology.orchestra.run.smithi053.stdout:2516 punch from 0xb61b545 to 0xb61f364, (0x3e1f bytes) 2024-09-13T05:02:35.074 INFO:teuthology.orchestra.run.smithi053.stdout:2519 writesame 0xa678ddb thru 0xa6834df data_size 0x1(0xa705 bytes) 2024-09-13T05:02:35.091 INFO:teuthology.orchestra.run.smithi053.stdout:2520 writesame 0x75a88a9 thru 0x75b153c data_size 0x6(0x8c94 bytes) 2024-09-13T05:02:35.097 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.097 INFO:teuthology.orchestra.run.smithi053.stdout:2523 read 0x77850fa thru 0x778f426 (0xa32d bytes) 2024-09-13T05:02:35.098 INFO:teuthology.orchestra.run.smithi053.stdout:2524 writesame 0xc626862 thru 0xc63119d data_size 0x2(0xa93c bytes) 2024-09-13T05:02:35.098 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:35.096+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:35.109 INFO:teuthology.orchestra.run.smithi053.stdout:2525 punch from 0x69aefe4 to 0x69b154c, (0x2568 bytes) 2024-09-13T05:02:35.109 INFO:teuthology.orchestra.run.smithi053.stdout:2527 writesame 0x138f6d7 thru 0x1393115 data_size 0x1f(0x3a3f bytes) 2024-09-13T05:02:35.115 INFO:teuthology.orchestra.run.smithi053.stdout:2528 read 0xa4e4041 thru 0xa4e7312 (0x32d2 bytes) 2024-09-13T05:02:35.115 INFO:teuthology.orchestra.run.smithi053.stdout:2532 read 0xd43158f thru 0xd432d05 (0x1777 bytes) 2024-09-13T05:02:35.115 INFO:teuthology.orchestra.run.smithi053.stdout:2533 read 0xcea370d thru 0xceaa665 (0x6f59 bytes) 2024-09-13T05:02:35.115 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.116 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.123 INFO:teuthology.orchestra.run.smithi053.stdout:2537 trunc from 0xe530660 to 0x5a86f4c 2024-09-13T05:02:35.138 INFO:teuthology.orchestra.run.smithi053.stdout:2538 trunc from 0x5a86f4c to 0x4e8448f 2024-09-13T05:02:35.149 INFO:teuthology.orchestra.run.smithi053.stdout:2539 read 0x40ea6ea thru 0x40f2482 (0x7d99 bytes) 2024-09-13T05:02:35.149 INFO:teuthology.orchestra.run.smithi053.stdout:2540 read 0x4dae7e4 thru 0x4daef4d (0x76a bytes) 2024-09-13T05:02:35.149 INFO:teuthology.orchestra.run.smithi053.stdout:2541 read 0xa48ae9 thru 0xa5563e (0xcb56 bytes) 2024-09-13T05:02:35.149 INFO:teuthology.orchestra.run.smithi053.stdout:2546 write 0x171d68c thru 0x1725406 (0x7d7b bytes) 2024-09-13T05:02:35.154 INFO:teuthology.orchestra.run.smithi053.stdout:2548 trunc from 0x4e8448f to 0x1f4001a 2024-09-13T05:02:35.162 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:35.162 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:35.166 INFO:teuthology.orchestra.run.smithi053.stdout:2549 punch from 0x170576c to 0x1710c9e, (0xb532 bytes) 2024-09-13T05:02:35.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:35.185 INFO:teuthology.orchestra.run.smithi053.stdout:2550 write 0xde3dd02 thru 0xde448d1 (0x6bd0 bytes) 2024-09-13T05:02:35.190 INFO:teuthology.orchestra.run.smithi053.stdout:2551 write 0xab08e3 thru 0xabb3bf (0xaadd bytes) 2024-09-13T05:02:35.194 INFO:teuthology.orchestra.run.smithi053.stdout:2555 read 0x2b97967 thru 0x2b9889d (0xf37 bytes) 2024-09-13T05:02:35.194 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.194 INFO:teuthology.orchestra.run.smithi053.stdout:2557 write 0xc75f278 thru 0xc76e827 (0xf5b0 bytes) 2024-09-13T05:02:35.198 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:35.197+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:35.199 INFO:teuthology.orchestra.run.smithi053.stdout:2558 read 0xa147530 thru 0xa14cc42 (0x5713 bytes) 2024-09-13T05:02:35.199 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.199 INFO:teuthology.orchestra.run.smithi053.stdout:2560 write 0xc4a626f thru 0xc4a837f (0x2111 bytes) 2024-09-13T05:02:35.202 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.202 INFO:teuthology.orchestra.run.smithi053.stdout:2562 write 0x9bb709f thru 0x9bbd897 (0x67f9 bytes) 2024-09-13T05:02:35.205 INFO:teuthology.orchestra.run.smithi053.stdout:2563 write 0x204e38d thru 0x205962f (0xb2a3 bytes) 2024-09-13T05:02:35.210 INFO:teuthology.orchestra.run.smithi053.stdout:2565 write 0xd633908 thru 0xd63cdb3 (0x94ac bytes) 2024-09-13T05:02:35.213 INFO:teuthology.orchestra.run.smithi053.stdout:2566 writesame 0x171d3d6 thru 0x1727079 data_size 0x32(0x9ca4 bytes) 2024-09-13T05:02:35.215 INFO:teuthology.orchestra.run.smithi053.stdout:2567 punch from 0xd48d4f3 to 0xd49ac49, (0xd756 bytes) 2024-09-13T05:02:35.215 INFO:teuthology.orchestra.run.smithi053.stdout:2568 write 0xae02d44 thru 0xae058da (0x2b97 bytes) 2024-09-13T05:02:35.219 INFO:teuthology.orchestra.run.smithi053.stdout:2569 read 0xa76a1a0 thru 0xa77124a (0x70ab bytes) 2024-09-13T05:02:35.226 INFO:teuthology.orchestra.run.smithi053.stdout:2570 write 0xea1d2fd thru 0xea2104b (0x3d4f bytes) 2024-09-13T05:02:35.230 INFO:teuthology.orchestra.run.smithi053.stdout:2571 write 0x8fe1f3d thru 0x8fee33d (0xc401 bytes) 2024-09-13T05:02:35.233 INFO:teuthology.orchestra.run.smithi053.stdout:2572 write 0x68a9395 thru 0x68b14ff (0x816b bytes) 2024-09-13T05:02:35.238 INFO:teuthology.orchestra.run.smithi053.stdout:2573 read 0x3bd6270 thru 0x3bdb244 (0x4fd5 bytes) 2024-09-13T05:02:35.238 INFO:teuthology.orchestra.run.smithi053.stdout:2574 write 0x95c0ddd thru 0x95c18b0 (0xad4 bytes) 2024-09-13T05:02:35.241 INFO:teuthology.orchestra.run.smithi053.stdout:2576 write 0xbab6b5a thru 0xbabcc9e (0x6145 bytes) 2024-09-13T05:02:35.244 INFO:teuthology.orchestra.run.smithi053.stdout:2579 write 0xb057f3d thru 0xb066ff3 (0xf0b7 bytes) 2024-09-13T05:02:35.248 INFO:teuthology.orchestra.run.smithi053.stdout:2580 writesame 0xdb4bfd3 thru 0xdb52f1c data_size 0x25(0x6f4a bytes) 2024-09-13T05:02:35.252 INFO:teuthology.orchestra.run.smithi053.stdout:2581 write 0x7a89ca9 thru 0x7a8ecdf (0x5037 bytes) 2024-09-13T05:02:35.255 INFO:teuthology.orchestra.run.smithi053.stdout:2582 write 0xd15a38b thru 0xd160486 (0x60fc bytes) 2024-09-13T05:02:35.260 INFO:teuthology.orchestra.run.smithi053.stdout:2583 write 0x4e32b96 thru 0x4e3d58b (0xa9f6 bytes) 2024-09-13T05:02:35.264 INFO:teuthology.orchestra.run.smithi053.stdout:2584 write 0x3525ca4 thru 0x352f675 (0x99d2 bytes) 2024-09-13T05:02:35.267 INFO:teuthology.orchestra.run.smithi053.stdout:2585 read 0xcd817a9 thru 0xcd90427 (0xec7f bytes) 2024-09-13T05:02:35.268 INFO:teuthology.orchestra.run.smithi053.stdout:2586 punch from 0x94e5cfe to 0x94eef8a, (0x928c bytes) 2024-09-13T05:02:35.268 INFO:teuthology.orchestra.run.smithi053.stdout:2587 writesame 0x2082521 thru 0x208c9e6 data_size 0xe(0xa4c6 bytes) 2024-09-13T05:02:35.278 INFO:teuthology.orchestra.run.smithi053.stdout:2589 trunc from 0xea2104c to 0x91767db 2024-09-13T05:02:35.292 INFO:teuthology.orchestra.run.smithi053.stdout:2592 read 0x7555630 thru 0x75592ee (0x3cbf bytes) 2024-09-13T05:02:35.295 INFO:teuthology.orchestra.run.smithi053.stdout:2593 trunc from 0x91767db to 0x68b1088 2024-09-13T05:02:35.299 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:35.299+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:35.303 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:35.303 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:35.313 INFO:teuthology.orchestra.run.smithi053.stdout:2594 writesame 0x6f77328 thru 0x6f79b70 data_size 0x1(0x2849 bytes) 2024-09-13T05:02:35.313 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.313 INFO:teuthology.orchestra.run.smithi053.stdout:2597 writesame 0x23b15a8 thru 0x23b5120 data_size 0x57(0x3b79 bytes) 2024-09-13T05:02:35.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:35.317 INFO:teuthology.orchestra.run.smithi053.stdout:2598 trunc from 0x6f79b71 to 0x643c1df 2024-09-13T05:02:35.340 INFO:teuthology.orchestra.run.smithi053.stdout:2599 write 0xbcd5826 thru 0xbcddac2 (0x829d bytes) 2024-09-13T05:02:35.344 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.344 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.344 INFO:teuthology.orchestra.run.smithi053.stdout:2602 punch from 0x83c1436 to 0x83cab94, (0x975e bytes) 2024-09-13T05:02:35.344 INFO:teuthology.orchestra.run.smithi053.stdout:2603 write 0x93101f1 thru 0x93119ee (0x17fe bytes) 2024-09-13T05:02:35.356 INFO:teuthology.orchestra.run.smithi053.stdout:2606 writesame 0xe532f30 thru 0xe53f947 data_size 0x8(0xca18 bytes) 2024-09-13T05:02:35.356 INFO:teuthology.orchestra.run.smithi053.stdout:2607 read 0x90537 thru 0x93782 (0x324c bytes) 2024-09-13T05:02:35.356 INFO:teuthology.orchestra.run.smithi053.stdout:2608 punch from 0x52883de to 0x5292f84, (0xaba6 bytes) 2024-09-13T05:02:35.356 INFO:teuthology.orchestra.run.smithi053.stdout:2609 read 0x1288d03 thru 0x12940c3 (0xb3c1 bytes) 2024-09-13T05:02:35.357 INFO:teuthology.orchestra.run.smithi053.stdout:2610 punch from 0x7367168 to 0x7371151, (0x9fe9 bytes) 2024-09-13T05:02:35.357 INFO:teuthology.orchestra.run.smithi053.stdout:2611 write 0xdb65b83 thru 0xdb6662b (0xaa9 bytes) 2024-09-13T05:02:35.360 INFO:teuthology.orchestra.run.smithi053.stdout:2612 trunc from 0xe53f948 to 0xdd713e1 2024-09-13T05:02:35.366 INFO:teuthology.orchestra.run.smithi053.stdout:2613 write 0x5270a33 thru 0x528089f (0xfe6d bytes) 2024-09-13T05:02:35.371 INFO:teuthology.orchestra.run.smithi053.stdout:2614 punch from 0xda3a9b9 to 0xda3b6cf, (0xd16 bytes) 2024-09-13T05:02:35.371 INFO:teuthology.orchestra.run.smithi053.stdout:2617 read 0x24ec2f thru 0x25e470 (0xf842 bytes) 2024-09-13T05:02:35.371 INFO:teuthology.orchestra.run.smithi053.stdout:2618 write 0x8ce71dc thru 0x8cedbfd (0x6a22 bytes) 2024-09-13T05:02:35.374 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.374 INFO:teuthology.orchestra.run.smithi053.stdout:2624 write 0xda95007 thru 0xdaa2647 (0xd641 bytes) 2024-09-13T05:02:35.377 INFO:teuthology.orchestra.run.smithi053.stdout:2625 read 0x915abd4 thru 0x9167926 (0xcd53 bytes) 2024-09-13T05:02:35.377 INFO:teuthology.orchestra.run.smithi053.stdout:2626 punch from 0xa809325 to 0xa80c9b8, (0x3693 bytes) 2024-09-13T05:02:35.377 INFO:teuthology.orchestra.run.smithi053.stdout:2628 punch from 0x6095589 to 0x609abe0, (0x5657 bytes) 2024-09-13T05:02:35.378 INFO:teuthology.orchestra.run.smithi053.stdout:2629 writesame 0xe445d thru 0xef99f data_size 0x7(0xb543 bytes) 2024-09-13T05:02:35.381 INFO:teuthology.orchestra.run.smithi053.stdout:2630 write 0x83818fb thru 0x838d0db (0xb7e1 bytes) 2024-09-13T05:02:35.384 INFO:teuthology.orchestra.run.smithi053.stdout:2631 writesame 0xc668253 thru 0xc66a825 data_size 0x1(0x25d3 bytes) 2024-09-13T05:02:35.390 INFO:teuthology.orchestra.run.smithi053.stdout:2632 punch from 0x8f4ea4e to 0x8f568c9, (0x7e7b bytes) 2024-09-13T05:02:35.390 INFO:teuthology.orchestra.run.smithi053.stdout:2633 punch from 0x107c540 to 0x1085ee0, (0x99a0 bytes) 2024-09-13T05:02:35.390 INFO:teuthology.orchestra.run.smithi053.stdout:2635 read 0x3c3161f thru 0x3c38127 (0x6b09 bytes) 2024-09-13T05:02:35.390 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.391 INFO:teuthology.orchestra.run.smithi053.stdout:2639 read 0x9446755 thru 0x944a78f (0x403b bytes) 2024-09-13T05:02:35.393 INFO:teuthology.orchestra.run.smithi053.stdout:2640 trunc from 0xdd713e1 to 0xb30a350 2024-09-13T05:02:35.400 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:35.399+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:35.403 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.403 INFO:teuthology.orchestra.run.smithi053.stdout:2642 read 0x1942e2b thru 0x1950b59 (0xdd2f bytes) 2024-09-13T05:02:35.403 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.403 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.404 INFO:teuthology.orchestra.run.smithi053.stdout:2645 read 0x7ac0e01 thru 0x7acd3df (0xc5df bytes) 2024-09-13T05:02:35.404 INFO:teuthology.orchestra.run.smithi053.stdout:2646 writesame 0x2bf33b7 thru 0x2bf64ad data_size 0x17(0x30f7 bytes) 2024-09-13T05:02:35.414 INFO:teuthology.orchestra.run.smithi053.stdout:2648 trunc from 0xb30a350 to 0x4e97c17 2024-09-13T05:02:35.428 INFO:teuthology.orchestra.run.smithi053.stdout:2649 punch from 0x116712c to 0x117007d, (0x8f51 bytes) 2024-09-13T05:02:35.436 INFO:teuthology.orchestra.run.smithi053.stdout:2650 write 0x764add2 thru 0x765092f (0x5b5e bytes) 2024-09-13T05:02:35.440 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.440 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:35.441 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:35.447 INFO:teuthology.orchestra.run.smithi053.stdout:2652 write 0x9271ade thru 0x92740cc (0x25ef bytes) 2024-09-13T05:02:35.451 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:35.458 INFO:teuthology.orchestra.run.smithi053.stdout:2653 trunc from 0x92740cd to 0x32d1ae3 2024-09-13T05:02:35.472 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:35.481 INFO:teuthology.orchestra.run.smithi053.stdout:2654 writesame 0x8598307 thru 0x85a66a8 data_size 0x1(0xe3a2 bytes) 2024-09-13T05:02:35.482 INFO:teuthology.orchestra.run.smithi053.stdout:2655 trunc from 0x85a66a9 to 0x77a8997 2024-09-13T05:02:35.493 INFO:teuthology.orchestra.run.smithi053.stdout:2657 trunc from 0x77a8997 to 0xc6027fe 2024-09-13T05:02:35.499 INFO:teuthology.orchestra.run.smithi053.stdout:2659 punch from 0x9ccd6bb to 0x9ccdcae, (0x5f3 bytes) 2024-09-13T05:02:35.500 INFO:teuthology.orchestra.run.smithi053.stdout:2660 punch from 0xbac0614 to 0xbac79aa, (0x7396 bytes) 2024-09-13T05:02:35.501 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:35.500+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:35.507 INFO:teuthology.orchestra.run.smithi053.stdout:2661 write 0xd55bb11 thru 0xd568de0 (0xd2d0 bytes) 2024-09-13T05:02:35.512 INFO:teuthology.orchestra.run.smithi053.stdout:2664 write 0x6e57966 thru 0x6e616eb (0x9d86 bytes) 2024-09-13T05:02:35.516 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.528 INFO:teuthology.orchestra.run.smithi053.stdout:2666 trunc from 0xd568de1 to 0x2a70735 2024-09-13T05:02:35.540 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.540 INFO:teuthology.orchestra.run.smithi053.stdout:2673 punch from 0x19df122 to 0x19e6248, (0x7126 bytes) 2024-09-13T05:02:35.540 INFO:teuthology.orchestra.run.smithi053.stdout:2674 read 0x1c06300 thru 0x1c0775f (0x1460 bytes) 2024-09-13T05:02:35.547 INFO:teuthology.orchestra.run.smithi053.stdout:2676 writesame 0x347a3db thru 0x347c673 data_size 0x1(0x2299 bytes) 2024-09-13T05:02:35.564 INFO:teuthology.orchestra.run.smithi053.stdout:2678 write 0xc32cfc2 thru 0xc32d01e (0x5d bytes) 2024-09-13T05:02:35.567 INFO:teuthology.orchestra.run.smithi053.stdout:2679 writesame 0x40f5567 thru 0x40f7432 data_size 0x24(0x1ecc bytes) 2024-09-13T05:02:35.575 INFO:teuthology.orchestra.run.smithi053.stdout:2680 writesame 0xc3e2847 thru 0xc3f04a9 data_size 0x17(0xdc63 bytes) 2024-09-13T05:02:35.577 INFO:teuthology.orchestra.run.smithi053.stdout:2681 read 0x275454f thru 0x275b8ad (0x735f bytes) 2024-09-13T05:02:35.585 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:35.585 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:35.586 INFO:teuthology.orchestra.run.smithi053.stdout:2682 trunc from 0xc3f04aa to 0x3a81420 2024-09-13T05:02:35.596 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:35.598 INFO:teuthology.orchestra.run.smithi053.stdout:2684 read 0x193cc19 thru 0x193f4f8 (0x28e0 bytes) 2024-09-13T05:02:35.612 INFO:teuthology.orchestra.run.smithi053.stdout:2688 writesame 0xa6ea61d thru 0xa6f370d data_size 0x29(0x90f1 bytes) 2024-09-13T05:02:35.612 INFO:teuthology.orchestra.run.smithi053.stdout:2689 write 0x4814a92 thru 0x4823c4a (0xf1b9 bytes) 2024-09-13T05:02:35.612 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:35.611+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:35.617 INFO:teuthology.orchestra.run.smithi053.stdout:2691 read 0x59dd043 thru 0x59e9735 (0xc6f3 bytes) 2024-09-13T05:02:35.617 INFO:teuthology.orchestra.run.smithi053.stdout:2692 punch from 0x7af24c6 to 0x7af7c4c, (0x5786 bytes) 2024-09-13T05:02:35.617 INFO:teuthology.orchestra.run.smithi053.stdout:2694 read 0x6a43fe thru 0x6b3092 (0xec95 bytes) 2024-09-13T05:02:35.618 INFO:teuthology.orchestra.run.smithi053.stdout:2697 write 0xc6ce16 thru 0xc76607 (0x97f2 bytes) 2024-09-13T05:02:35.622 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.622 INFO:teuthology.orchestra.run.smithi053.stdout:2700 write 0x9e66b88 thru 0x9e682f9 (0x1772 bytes) 2024-09-13T05:02:35.625 INFO:teuthology.orchestra.run.smithi053.stdout:2701 write 0xa560616 thru 0xa56b52b (0xaf16 bytes) 2024-09-13T05:02:35.639 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:35.639 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:35.639 INFO:teuthology.orchestra.run.smithi053.stdout:2703 write 0xdaba7b5 thru 0xdabe4fe (0x3d4a bytes) 2024-09-13T05:02:35.643 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.643 INFO:teuthology.orchestra.run.smithi053.stdout:2705 read 0xc0bfd19 thru 0xc0c65ea (0x68d2 bytes) 2024-09-13T05:02:35.646 INFO:teuthology.orchestra.run.smithi053.stdout:2707 trunc from 0xdabe4ff to 0xb325a6a 2024-09-13T05:02:35.657 INFO:teuthology.orchestra.run.smithi053.stdout:2708 punch from 0x8bacaab to 0x8bb2135, (0x568a bytes) 2024-09-13T05:02:35.657 INFO:teuthology.orchestra.run.smithi053.stdout:2709 writesame 0x42c95bd thru 0x42d79ec data_size 0x18(0xe430 bytes) 2024-09-13T05:02:35.662 INFO:teuthology.orchestra.run.smithi053.stdout:2711 read 0x9b51278 thru 0x9b59fa4 (0x8d2d bytes) 2024-09-13T05:02:35.662 INFO:teuthology.orchestra.run.smithi053.stdout:2712 read 0xa9c9c5 thru 0xaa74c8 (0xab04 bytes) 2024-09-13T05:02:35.662 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.662 INFO:teuthology.orchestra.run.smithi053.stdout:2715 read 0x80c8aad thru 0x80ce16b (0x56bf bytes) 2024-09-13T05:02:35.662 INFO:teuthology.orchestra.run.smithi053.stdout:2716 read 0x4f3a92f thru 0x4f42578 (0x7c4a bytes) 2024-09-13T05:02:35.662 INFO:teuthology.orchestra.run.smithi053.stdout:2717 write 0x322735c thru 0x322bba0 (0x4845 bytes) 2024-09-13T05:02:35.663 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:35.663 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:35.665 INFO:teuthology.orchestra.run.smithi053.stdout:2718 read 0x2dd1b9e thru 0x2ddc092 (0xa4f5 bytes) 2024-09-13T05:02:35.666 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.666 INFO:teuthology.orchestra.run.smithi053.stdout:2720 punch from 0x4ecbc36 to 0x4ed635a, (0xa724 bytes) 2024-09-13T05:02:35.666 INFO:teuthology.orchestra.run.smithi053.stdout:2722 punch from 0x7908b2 to 0x791233, (0x981 bytes) 2024-09-13T05:02:35.674 INFO:teuthology.orchestra.run.smithi053.stdout:2723 write 0xc7bd6aa thru 0xc7c4511 (0x6e68 bytes) 2024-09-13T05:02:35.678 INFO:teuthology.orchestra.run.smithi053.stdout:2724 read 0x5e89153 thru 0x5e91b61 (0x8a0f bytes) 2024-09-13T05:02:35.678 INFO:teuthology.orchestra.run.smithi053.stdout:2725 write 0x62f2fe thru 0x63de46 (0xeb49 bytes) 2024-09-13T05:02:35.681 INFO:teuthology.orchestra.run.smithi053.stdout:2727 punch from 0x799dda to 0x7a264f, (0x8875 bytes) 2024-09-13T05:02:35.681 INFO:teuthology.orchestra.run.smithi053.stdout:2728 write 0x6e996b2 thru 0x6ea2607 (0x8f56 bytes) 2024-09-13T05:02:35.685 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:35.685 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:35.686 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.686 INFO:teuthology.orchestra.run.smithi053.stdout:2731 writesame 0x215b538 thru 0x215e952 data_size 0x1(0x341b bytes) 2024-09-13T05:02:35.693 INFO:teuthology.orchestra.run.smithi053.stdout:2732 write 0x8eba997 thru 0x8ec3755 (0x8dbf bytes) 2024-09-13T05:02:35.698 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.698 INFO:teuthology.orchestra.run.smithi053.stdout:2736 read 0x5989c93 thru 0x598f312 (0x5680 bytes) 2024-09-13T05:02:35.698 INFO:teuthology.orchestra.run.smithi053.stdout:2738 punch from 0x7beea33 to 0x7bf3a4c, (0x5019 bytes) 2024-09-13T05:02:35.699 INFO:teuthology.orchestra.run.smithi053.stdout:2739 write 0x254e5cb thru 0x2559f9c (0xb9d2 bytes) 2024-09-13T05:02:35.702 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:35.701+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:35.703 INFO:teuthology.orchestra.run.smithi053.stdout:2741 writesame 0xbd3753f thru 0xbd46b7d data_size 0x1(0xf63f bytes) 2024-09-13T05:02:35.706 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:35.721 INFO:teuthology.orchestra.run.smithi053.stdout:2742 write 0x7417ec2 thru 0x74236fb (0xb83a bytes) 2024-09-13T05:02:35.725 INFO:teuthology.orchestra.run.smithi053.stdout:2743 punch from 0x394f6a6 to 0x39599a4, (0xa2fe bytes) 2024-09-13T05:02:35.726 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:35.726 INFO:teuthology.orchestra.run.smithi053.stdout:2745 write 0x6cd4e1b thru 0x6cdd6fc (0x88e2 bytes) 2024-09-13T05:02:35.727 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:35.727 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:35.728 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:35.729 INFO:teuthology.orchestra.run.smithi053.stdout:2746 punch from 0x9182c00 to 0x918d6d0, (0xaad0 bytes) 2024-09-13T05:02:35.729 INFO:teuthology.orchestra.run.smithi053.stdout:2748 punch from 0xc03ac92 to 0xc042151, (0x74bf bytes) 2024-09-13T05:02:35.729 INFO:teuthology.orchestra.run.smithi053.stdout:2749 read 0xb50923f thru 0xb516e22 (0xdbe4 bytes) 2024-09-13T05:02:35.729 INFO:teuthology.orchestra.run.smithi053.stdout:2750 clone 16 order 23 su 131072 sc 9 2024-09-13T05:02:35.731 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:35.731 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:35.737 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:35.748 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:35.748 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:35.764 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:35.764 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:35.781 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:35.781 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:35.833 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:35.833+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:35.856 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:35.856 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:35.856 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:35.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:35.905 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:35.905+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:35.974 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:02:36.003 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:36.003+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:36.019 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32275 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7995392~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2382388189999993, 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010461302, 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32275 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780087+0000", 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780089+0000", 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2010000000000002e-06 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780095+0000", 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8019999999999997e-06 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780099+0000", 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3229999999999999e-06 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790415+0000", 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010315649 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790475+0000", 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9843999999999998e-05 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790549+0000", 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3883999999999999e-05 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32276 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8224768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2382230500000002, 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010619966, 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32276 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780103+0000", 2024-09-13T05:02:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780105+0000", 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.159e-06 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780111+0000", 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5639999999999996e-06 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780118+0000", 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8259999999999993e-06 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790585+0000", 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010466197 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790648+0000", 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3927e-05 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790723+0000", 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4635999999999995e-05 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32277 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8454144~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2382040790000008, 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010784396999999999, 2024-09-13T05:02:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32277 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780122+0000", 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780124+0000", 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.119e-06 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780129+0000", 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5799999999999999e-06 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780134+0000", 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4750000000000004e-06 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790750+0000", 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010616082000000001 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790824+0000", 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4031999999999999e-05 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790907+0000", 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2458000000000002e-05 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32278 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8683520~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2381611049999997, 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010917582, 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32278 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780165+0000", 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780164+0000", 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780169+0000", 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8300000000000003e-06 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780179+0000", 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0164e-05 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780186+0000", 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4799999999999998e-06 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790933+0000", 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010747358 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791001+0000", 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7390000000000004e-05 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791083+0000", 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2177999999999998e-05 2024-09-13T05:02:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32279 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8912896~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2381350510000004, 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011066273999999999, 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32279 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780191+0000", 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780193+0000", 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780200+0000", 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3709999999999996e-06 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780205+0000", 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.13e-06 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791108+0000", 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010903375 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791175+0000", 2024-09-13T05:02:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6451999999999993e-05 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791257+0000", 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2645999999999994e-05 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32317 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8945664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.237978837, 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010478401, 2024-09-13T05:02:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32317 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780347+0000", 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780350+0000", 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9790000000000002e-06 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780357+0000", 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2350000000000002e-06 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780362+0000", 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3809999999999998e-06 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790666+0000", 2024-09-13T05:02:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010303911000000001 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790735+0000", 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8396e-05 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790826+0000", 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0983999999999995e-05 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32318 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9175040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2379591160000007, 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010623140999999999, 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32318 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780367+0000", 2024-09-13T05:02:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780369+0000", 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5620000000000002e-06 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780375+0000", 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780380+0000", 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6879999999999998e-06 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790855+0000", 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010474957 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790920+0000", 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5030999999999995e-05 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.790990+0000", 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0384999999999998e-05 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32337 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7569408~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.237929437, 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011667398000000001, 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32337 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780397+0000", 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780396+0000", 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780399+0000", 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9849999999999998e-06 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780406+0000", 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.878e-06 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780415+0000", 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6799999999999999e-06 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791286+0000", 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010870869999999999 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791955+0000", 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00066933100000000001 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792064+0000", 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010907499999999999 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32338 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 7798784~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2379063769999998, 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011830083, 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32338 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780420+0000", 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780419+0000", 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780422+0000", 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.339e-06 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780428+0000", 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0730000000000001e-06 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780433+0000", 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7010000000000004e-06 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792095+0000", 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011662529 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792164+0000", 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9346999999999999e-05 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792250+0000", 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5504e-05 2024-09-13T05:02:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32339 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8028160~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2378894079999991, 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011996256, 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32339 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780437+0000", 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780436+0000", 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780439+0000", 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.198e-06 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780444+0000", 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.806e-06 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780451+0000", 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4099999999999996e-06 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792275+0000", 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011824573999999999 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792339+0000", 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3628999999999997e-05 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792433+0000", 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4029000000000004e-05 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32340 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8257536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2378713549999993, 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012155473999999999, 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32340 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780455+0000", 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780457+0000", 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0609999999999998e-06 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780462+0000", 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7490000000000004e-06 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780467+0000", 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5970000000000002e-06 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792461+0000", 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011993668000000001 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792529+0000", 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8249000000000003e-05 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792610+0000", 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1451000000000003e-05 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32341 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8486912~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2378550009999998, 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012301632999999999, 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32341 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780471+0000", 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780473+0000", 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3269999999999999e-06 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780481+0000", 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7970000000000001e-06 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780485+0000", 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.408e-06 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792636+0000", 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012150879 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792699+0000", 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2842000000000006e-05 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792773+0000", 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3730000000000001e-05 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32342 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8716288~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2378360879999999, 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012452926, 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32342 2024-09-13T05:02:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780490+0000", 2024-09-13T05:02:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780493+0000", 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9979999999999999e-06 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780499+0000", 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5400000000000001e-06 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780504+0000", 2024-09-13T05:02:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6909999999999996e-06 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792805+0000", 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012301288 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792868+0000", 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3202999999999998e-05 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792943+0000", 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4592999999999995e-05 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32343 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 8945664~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2378172369999998, 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012601064, 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32343 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780509+0000", 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780511+0000", 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.215e-06 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780516+0000", 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6380000000000001e-06 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780521+0000", 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7770000000000002e-06 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792968+0000", 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012446262 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793031+0000", 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3609999999999996e-05 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793110+0000", 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8910000000000002e-05 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32344 3.4 3:203fc763:::rbd_data.117c64fb1ed3.0000000000000001:f [read 9175040~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2378006859999999, 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012769502, 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32344 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780526+0000", 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780525+0000", 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780527+0000", 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3649999999999998e-06 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780534+0000", 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5980000000000002e-06 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.780538+0000", 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.194e-06 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793139+0000", 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012600731 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793210+0000", 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1131999999999996e-05 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793295+0000", 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4938000000000006e-05 2024-09-13T05:02:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32379 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8749056~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2367605699999995, 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010495398, 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32379 2024-09-13T05:02:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781566+0000", 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781565+0000", 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781568+0000", 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781574+0000", 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2929999999999997e-06 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781579+0000", 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0340000000000004e-06 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791907+0000", 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010328027 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.791977+0000", 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9728999999999996e-05 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792061+0000", 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4332000000000005e-05 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32380 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 8978432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2367427549999999, 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010658737, 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32380 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781583+0000", 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781585+0000", 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0279999999999999e-06 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781589+0000", 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781591+0000", 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6120000000000001e-06 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792087+0000", 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010496152 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792161+0000", 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3473999999999995e-05 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792242+0000", 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1403999999999995e-05 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32381 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9207808~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2367328989999997, 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012026156, 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32381 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781593+0000", 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781594+0000", 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3850000000000001e-06 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781597+0000", 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9579999999999999e-06 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781600+0000", 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4169999999999999e-06 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792270+0000", 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010670059000000001 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.792339+0000", 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8861000000000004e-05 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793619+0000", 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001280712 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:32382 3.0 3:068253e6:::rbd_data.117c64fb1ed3.0000000000000005:f [read 9437184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2367238680000003, 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012273719000000001, 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 32382 2024-09-13T05:02:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781602+0000", 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781604+0000", 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.905e-06 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781610+0000", 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3799999999999999e-06 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.781614+0000", 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5870000000000001e-06 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793687+0000", 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012073084 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793763+0000", 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6074000000000004e-05 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.793876+0000", 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011301199999999999 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:40343 3.4 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.936337+0000", 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.081988881, 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011182364, 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 40343 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936337+0000", 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936337+0000", 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936336+0000", 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936345+0000", 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9409999999999996e-06 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936362+0000", 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7144000000000001e-05 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936372+0000", 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7950000000000002e-06 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936436+0000", 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4688000000000001e-05 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.936587+0000", 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00015060400000000001 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 2", 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.937505+0000", 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00091796800000000002 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947390+0000", 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098854700000000004 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947432+0000", 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1678999999999999e-05 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947479+0000", 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7119000000000001e-05 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947520+0000", 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0646999999999999e-05 2024-09-13T05:02:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.109 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:36.110 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:36.111 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:36.109+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:36.173 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-13T05:02:36.200 INFO:teuthology.orchestra.run.smithi053.stdout:2751 write 0x708f361 thru 0x709ccf5 (0xd995 bytes) 2024-09-13T05:02:36.209 INFO:teuthology.orchestra.run.smithi053.stdout:2752 punch from 0x36b029b to 0x36b8cc6, (0x8a2b bytes) 2024-09-13T05:02:36.209 INFO:teuthology.orchestra.run.smithi053.stdout:2753 write 0x1e8d85d thru 0x1e95219 (0x79bd bytes) 2024-09-13T05:02:36.210 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:36.210+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:36.214 INFO:teuthology.orchestra.run.smithi053.stdout:2754 read 0xb383812 thru 0xb38fc4e (0xc43d bytes) 2024-09-13T05:02:36.222 INFO:teuthology.orchestra.run.smithi053.stdout:2755 trunc from 0xc7c4512 to 0x56c50c1 2024-09-13T05:02:36.244 INFO:teuthology.orchestra.run.smithi053.stdout:2758 write 0x637067e thru 0x6371eec (0x186f bytes) 2024-09-13T05:02:36.246 INFO:teuthology.orchestra.run.smithi053.stdout:2760 punch from 0x179f44 to 0x17ce28, (0x2ee4 bytes) 2024-09-13T05:02:36.246 INFO:teuthology.orchestra.run.smithi053.stdout:2761 punch from 0x5ec1a3d to 0x5ecf23e, (0xd801 bytes) 2024-09-13T05:02:36.246 INFO:teuthology.orchestra.run.smithi053.stdout:2762 punch from 0x1c1d745 to 0x1c2d3d1, (0xfc8c bytes) 2024-09-13T05:02:36.246 INFO:teuthology.orchestra.run.smithi053.stdout:2763 read 0x35b0cc thru 0x35bf9f (0xed4 bytes) 2024-09-13T05:02:36.259 INFO:teuthology.orchestra.run.smithi053.stdout:2765 write 0xd7d3477 thru 0xd7dba82 (0x860c bytes) 2024-09-13T05:02:36.263 INFO:teuthology.orchestra.run.smithi053.stdout:2766 write 0xb3daa6c thru 0xb3e3db4 (0x9349 bytes) 2024-09-13T05:02:36.265 INFO:teuthology.orchestra.run.smithi053.stdout:2769 write 0x8666905 thru 0x866d61b (0x6d17 bytes) 2024-09-13T05:02:36.267 INFO:teuthology.orchestra.run.smithi053.stdout:2770 read 0xd0878af thru 0xd0884fa (0xc4c bytes) 2024-09-13T05:02:36.267 INFO:teuthology.orchestra.run.smithi053.stdout:2771 read 0x73501a4 thru 0x7351353 (0x11b0 bytes) 2024-09-13T05:02:36.267 INFO:teuthology.orchestra.run.smithi053.stdout:2772 punch from 0x5c5f3a0 to 0x5c67283, (0x7ee3 bytes) 2024-09-13T05:02:36.268 INFO:teuthology.orchestra.run.smithi053.stdout:2773 read 0x29a321b thru 0x29aca4d (0x9833 bytes) 2024-09-13T05:02:36.268 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.275 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:36.276 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:36.276 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:36.278 INFO:teuthology.orchestra.run.smithi053.stdout:2775 trunc from 0xd7dba83 to 0x403a7dc 2024-09-13T05:02:36.289 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:36.306 INFO:teuthology.orchestra.run.smithi053.stdout:2777 punch from 0x14a02ac to 0x14aa3d4, (0xa128 bytes) 2024-09-13T05:02:36.306 INFO:teuthology.orchestra.run.smithi053.stdout:2778 write 0x29a56ad thru 0x29b3961 (0xe2b5 bytes) 2024-09-13T05:02:36.308 INFO:teuthology.orchestra.run.smithi053.stdout:2781 read 0x2981ce2 thru 0x298ba45 (0x9d64 bytes) 2024-09-13T05:02:36.308 INFO:teuthology.orchestra.run.smithi053.stdout:2782 read 0x583603 thru 0x5920d9 (0xead7 bytes) 2024-09-13T05:02:36.309 INFO:teuthology.orchestra.run.smithi053.stdout:2783 write 0x3c9b2d2 thru 0x3ca427b (0x8faa bytes) 2024-09-13T05:02:36.311 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:36.310+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:36.328 INFO:teuthology.orchestra.run.smithi053.stdout:2784 write 0xece6498 thru 0xecea2b8 (0x3e21 bytes) 2024-09-13T05:02:36.343 INFO:teuthology.orchestra.run.smithi053.stdout:2786 trunc from 0xecea2b9 to 0x42ed83f 2024-09-13T05:02:36.354 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.354 INFO:teuthology.orchestra.run.smithi053.stdout:2788 write 0x39422b0 thru 0x3945690 (0x33e1 bytes) 2024-09-13T05:02:36.362 INFO:teuthology.orchestra.run.smithi053.stdout:2789 write 0x671ac64 thru 0x67265b8 (0xb955 bytes) 2024-09-13T05:02:36.380 INFO:teuthology.orchestra.run.smithi053.stdout:2791 writesame 0xde1bed9 thru 0xde1c936 data_size 0x1(0xa5e bytes) 2024-09-13T05:02:36.380 INFO:teuthology.orchestra.run.smithi053.stdout:2792 write 0x639fdca thru 0x63a0acb (0xd02 bytes) 2024-09-13T05:02:36.383 INFO:teuthology.orchestra.run.smithi053.stdout:2793 read 0x1d2f42c thru 0x1d3d6ef (0xe2c4 bytes) 2024-09-13T05:02:36.388 INFO:teuthology.orchestra.run.smithi053.stdout:2794 write 0xe032070 thru 0xe03bf30 (0x9ec1 bytes) 2024-09-13T05:02:36.391 INFO:teuthology.orchestra.run.smithi053.stdout:2795 read 0x3b47812 thru 0x3b50eff (0x96ee bytes) 2024-09-13T05:02:36.391 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.391 INFO:teuthology.orchestra.run.smithi053.stdout:2798 write 0x278131a thru 0x278a888 (0x956f bytes) 2024-09-13T05:02:36.393 INFO:teuthology.orchestra.run.smithi053.stdout:2799 write 0x2bd7bfe thru 0x2bdbfc6 (0x43c9 bytes) 2024-09-13T05:02:36.394 INFO:teuthology.orchestra.run.smithi053.stdout:2800 read 0x6b6e1ee thru 0x6b6f435 (0x1248 bytes) 2024-09-13T05:02:36.395 INFO:teuthology.orchestra.run.smithi053.stdout:2801 read 0x8990422 thru 0x8997c22 (0x7801 bytes) 2024-09-13T05:02:36.395 INFO:teuthology.orchestra.run.smithi053.stdout:2805 read 0x1c27128 thru 0x1c2f797 (0x8670 bytes) 2024-09-13T05:02:36.395 INFO:teuthology.orchestra.run.smithi053.stdout:2806 write 0x8b6f359 thru 0x8b7c8ad (0xd555 bytes) 2024-09-13T05:02:36.398 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.398 INFO:teuthology.orchestra.run.smithi053.stdout:2811 read 0xa9acb5b thru 0xa9b45ee (0x7a94 bytes) 2024-09-13T05:02:36.410 INFO:teuthology.orchestra.run.smithi053.stdout:2812 trunc from 0xe03bf31 to 0x3b84f8e 2024-09-13T05:02:36.411 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:36.410+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:36.422 INFO:teuthology.orchestra.run.smithi053.stdout:2813 read 0xbbeb52 thru 0xbc8b85 (0xa034 bytes) 2024-09-13T05:02:36.423 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.423 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.423 INFO:teuthology.orchestra.run.smithi053.stdout:2816 punch from 0x37052c to 0x37d75a, (0xd22e bytes) 2024-09-13T05:02:36.423 INFO:teuthology.orchestra.run.smithi053.stdout:2817 read 0x6ec940 thru 0x6f5bbe (0x927f bytes) 2024-09-13T05:02:36.423 INFO:teuthology.orchestra.run.smithi053.stdout:2818 punch from 0x203f399 to 0x204a0e6, (0xad4d bytes) 2024-09-13T05:02:36.428 INFO:teuthology.orchestra.run.smithi053.stdout:2819 write 0x3fffb0f thru 0x4006681 (0x6b73 bytes) 2024-09-13T05:02:36.430 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:36.430 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:36.430 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:36.431 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:36.431 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:36.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:36.446 INFO:teuthology.orchestra.run.smithi053.stdout:2820 write 0xd796aa9 thru 0xd79e985 (0x7edd bytes) 2024-09-13T05:02:36.450 INFO:teuthology.orchestra.run.smithi053.stdout:2821 read 0x9b7f41d thru 0x9b800eb (0xccf bytes) 2024-09-13T05:02:36.450 INFO:teuthology.orchestra.run.smithi053.stdout:2823 read 0x97ae39d thru 0x97afdc3 (0x1a27 bytes) 2024-09-13T05:02:36.450 INFO:teuthology.orchestra.run.smithi053.stdout:2824 write 0x7c69853 thru 0x7c6b327 (0x1ad5 bytes) 2024-09-13T05:02:36.459 INFO:teuthology.orchestra.run.smithi053.stdout:2825 writesame 0xe329bd7 thru 0xe32c652 data_size 0x2(0x2a7c bytes) 2024-09-13T05:02:36.464 INFO:teuthology.orchestra.run.smithi053.stdout:2829 write 0xead30e8 thru 0xead9f0d (0x6e26 bytes) 2024-09-13T05:02:36.478 INFO:teuthology.orchestra.run.smithi053.stdout:2830 trunc from 0xead9f0e to 0x4b3b8f5 2024-09-13T05:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout:2832 read 0x3590f4c thru 0x359e1f1 (0xd2a6 bytes) 2024-09-13T05:02:36.488 INFO:teuthology.orchestra.run.smithi053.stdout:2833 write 0x44724ae thru 0x447bc45 (0x9798 bytes) 2024-09-13T05:02:36.490 INFO:teuthology.orchestra.run.smithi053.stdout:2834 read 0x4030668 thru 0x40401f0 (0xfb89 bytes) 2024-09-13T05:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout:2835 writesame 0xa387028 thru 0xa38cfc4 data_size 0x29(0x5f9d bytes) 2024-09-13T05:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.502 INFO:teuthology.orchestra.run.smithi053.stdout:2838 punch from 0x3195833 to 0x31a3a38, (0xe205 bytes) 2024-09-13T05:02:36.503 INFO:teuthology.orchestra.run.smithi053.stdout:2839 trunc from 0xa38cfc5 to 0x8ee5118 2024-09-13T05:02:36.510 INFO:teuthology.orchestra.run.smithi053.stdout:2841 write 0x71ee7a2 thru 0x71fc4be (0xdd1d bytes) 2024-09-13T05:02:36.512 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:36.512+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:36.513 INFO:teuthology.orchestra.run.smithi053.stdout:2843 write 0x3a85119 thru 0x3a8d93c (0x8824 bytes) 2024-09-13T05:02:36.514 INFO:teuthology.orchestra.run.smithi053.stdout:2845 read 0x3b80984 thru 0x3b8b2e6 (0xa963 bytes) 2024-09-13T05:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout:2847 read 0x5e6358 thru 0x5ea114 (0x3dbd bytes) 2024-09-13T05:02:36.515 INFO:teuthology.orchestra.run.smithi053.stdout:2848 read 0x6d58258 thru 0x6d67eb1 (0xfc5a bytes) 2024-09-13T05:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout:2850 read 0x1aabe27 thru 0x1ab857a (0xc754 bytes) 2024-09-13T05:02:36.516 INFO:teuthology.orchestra.run.smithi053.stdout:2851 read 0x667e6b3 thru 0x668164a (0x2f98 bytes) 2024-09-13T05:02:36.523 INFO:teuthology.orchestra.run.smithi053.stdout:2852 write 0xa202a70 thru 0xa204746 (0x1cd7 bytes) 2024-09-13T05:02:36.526 INFO:teuthology.orchestra.run.smithi053.stdout:2853 write 0x70edc2b thru 0x70f4507 (0x68dd bytes) 2024-09-13T05:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout:2855 read 0x3dd2d71 thru 0x3dd6fbf (0x424f bytes) 2024-09-13T05:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout:2856 read 0x8086d54 thru 0x8091e52 (0xb0ff bytes) 2024-09-13T05:02:36.529 INFO:teuthology.orchestra.run.smithi053.stdout:2857 write 0x38320a6 thru 0x3837b1d (0x5a78 bytes) 2024-09-13T05:02:36.531 INFO:teuthology.orchestra.run.smithi053.stdout:2858 write 0x34d9921 thru 0x34de030 (0x4710 bytes) 2024-09-13T05:02:36.533 INFO:teuthology.orchestra.run.smithi053.stdout:2859 write 0x9753d05 thru 0x9759687 (0x5983 bytes) 2024-09-13T05:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout:2860 punch from 0x4f9b438 to 0x4f9ce66, (0x1a2e bytes) 2024-09-13T05:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout:2861 punch from 0x8c1786c to 0x8c1cc37, (0x53cb bytes) 2024-09-13T05:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout:2864 read 0x24f8cd4 thru 0x24fb506 (0x2833 bytes) 2024-09-13T05:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.536 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.537 INFO:teuthology.orchestra.run.smithi053.stdout:2867 trunc from 0xa204747 to 0x96ca319 2024-09-13T05:02:36.543 INFO:teuthology.orchestra.run.smithi053.stdout:2868 write 0x6e3383e thru 0x6e3ccbc (0x947f bytes) 2024-09-13T05:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout:2870 read 0x4e59fd thru 0x4f16c4 (0xbcc8 bytes) 2024-09-13T05:02:36.546 INFO:teuthology.orchestra.run.smithi053.stdout:2871 read 0x82c4212 thru 0x82ce467 (0xa256 bytes) 2024-09-13T05:02:36.547 INFO:teuthology.orchestra.run.smithi053.stdout:2875 read 0x386649a thru 0x38664a4 (0xb bytes) 2024-09-13T05:02:36.556 INFO:teuthology.orchestra.run.smithi053.stdout:2876 trunc from 0x96ca319 to 0x11e948c 2024-09-13T05:02:36.568 INFO:teuthology.orchestra.run.smithi053.stdout:2877 read 0x39b874 thru 0x3aa01a (0xe7a7 bytes) 2024-09-13T05:02:36.569 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:9144 3.4 e28/19 3:21949514:::rbd_data.117c6c62fa22.0000000000000052:head v 28'73, mlcod=27'71)", 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655502239, 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001561832, 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912419+0000", 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8749999999999999e-06 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912432+0000", 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2373e-05 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912440+0000", 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2889999999999998e-06 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913219+0000", 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077888000000000002 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913240+0000", 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0672000000000001e-05 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913941+0000", 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070157300000000004 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913977+0000", 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5472000000000002e-05 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.589853146999999, 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:02:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10094 3.4 e29/19 3:24263961:::rbd_data.117c6c62fa22.000000000000000b:head v 29'95, mlcod=28'92)", 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.586781737999999, 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001767984, 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981152+0000", 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6986999999999998e-05 2024-09-13T05:02:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981166+0000", 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4331000000000001e-05 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981177+0000", 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0858e-05 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981233+0000", 2024-09-13T05:02:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5813000000000002e-05 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981253+0000", 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9707999999999998e-05 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982869+0000", 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001615932 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982903+0000", 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4626999999999999e-05 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10121 3.4 e29/19 3:2672cf5a:::rbd_data.117c6c62fa22.0000000000000029:head v 29'97, mlcod=28'93)", 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.989656+0000", 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.578260821000001, 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001391357, 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989656+0000", 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989656+0000", 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989655+0000", 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989663+0000", 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8760000000000005e-06 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989677+0000", 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.398e-05 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989684+0000", 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8600000000000004e-06 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989733+0000", 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8950999999999999e-05 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.989755+0000", 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2008999999999999e-05 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.991015+0000", 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012602970000000001 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.991048+0000", 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2249000000000002e-05 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10141 3.4 e29/19 3:24508b78:::rbd_data.117c6c62fa22.000000000000003d:head v 29'101, mlcod=29'97)", 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.573039025, 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001501142, 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994877+0000", 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994885+0000", 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-09-13T05:02:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994900+0000", 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4698999999999999e-05 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994907+0000", 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2570000000000001e-06 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994952+0000", 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4885000000000001e-05 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994971+0000", 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9093000000000002e-05 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996346+0000", 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001375287 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996379+0000", 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2700999999999997e-05 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10143 3.0 e29/19 3:0297533c:::rbd_data.117c6c62fa22.000000000000003f:head v 29'99, mlcod=28'93)", 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.572546857999999, 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001524173, 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995375+0000", 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8340000000000004e-06 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995387+0000", 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1046000000000001e-05 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995393+0000", 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995441+0000", 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7786e-05 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995460+0000", 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9154e-05 2024-09-13T05:02:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996857+0000", 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013971719999999999 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996894+0000", 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7357999999999997e-05 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:19907 3.9 e31/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 31'115, mlcod=31'112)", 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.121072988, 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001592883, 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446843+0000", 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446850+0000", 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3450000000000002e-06 2024-09-13T05:02:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446860+0000", 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446867+0000", 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3570000000000003e-06 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446923+0000", 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6242000000000001e-05 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446940+0000", 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7209000000000001e-05 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448406+0000", 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001465383 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448437+0000", 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1372999999999999e-05 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:20095 3.9 e32/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 32'140, mlcod=32'137)", 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.372089226, 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014673279999999999, 2024-09-13T05:02:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195827+0000", 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195846+0000", 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8810000000000001e-05 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195860+0000", 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3835000000000001e-05 2024-09-13T05:02:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195868+0000", 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2300000000000008e-06 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195910+0000", 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1690000000000002e-05 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195938+0000", 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8829000000000001e-05 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.197231+0000", 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001292201 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.197295+0000", 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4498000000000005e-05 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:20115 3.0 e32/19 3:07e79267:::rbd_data.117c3521aa89.000000000000000a:head v 32'193, mlcod=31'190)", 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.365298492999999, 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015367919999999999, 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202618+0000", 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202626+0000", 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0700000000000007e-06 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202636+0000", 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0227000000000001e-05 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202645+0000", 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8040000000000007e-06 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202687+0000", 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2360000000000001e-05 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202706+0000", 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8933e-05 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.204117+0000", 2024-09-13T05:02:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014105089999999999 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.204155+0000", 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8544999999999998e-05 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.7644690920000006, 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.6234000579999996, 2024-09-13T05:02:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:02:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40489 3.4 e37/19 3:234d8752:::rbd_data.117c4a06f7bc.000000000000001a:head v 37'216, mlcod=37'214)", 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.5040616760000001, 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014028300000000001, 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063855+0000", 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063866+0000", 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1114000000000001e-05 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063873+0000", 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2180000000000002e-06 2024-09-13T05:02:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.063879+0000", 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2530000000000004e-06 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.064215+0000", 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033607 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.064232+0000", 2024-09-13T05:02:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.768e-05 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.065228+0000", 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00099559299999999991 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.065258+0000", 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0261999999999999e-05 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3918794080000003, 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:02:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3909037350000002, 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3905828580000001, 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:02:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3884949009999996, 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:02:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40675 3.4 e37/19 3:25c10db2:::rbd_data.117c4a06f7bc.000000000000024b:head v 37'226, mlcod=37'224)", 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:36.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3847882890000003, 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001577501, 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183128+0000", 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183134+0000", 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.387e-06 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183149+0000", 2024-09-13T05:02:36.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4664000000000001e-05 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183156+0000", 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9700000000000002e-06 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183436+0000", 2024-09-13T05:02:36.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027994199999999998 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183455+0000", 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9287999999999999e-05 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184676+0000", 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001220308 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184706+0000", 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639000000000003e-05 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40683 3.4 e37/19 3:2071a6c5:::rbd_data.117c4a06f7bc.000000000000025d:head v 37'227, mlcod=37'225)", 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.3826251459999996, 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001476371, 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185291+0000", 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185298+0000", 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185313+0000", 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5406000000000001e-05 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185326+0000", 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2313999999999999e-05 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185431+0000", 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010521799999999999 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185450+0000", 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9315000000000001e-05 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.186710+0000", 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012597730000000001 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.186768+0000", 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8236000000000003e-05 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41018 3.0 e38/19 3:004a91c0:::rbd_data.117c4a06f7bc.0000000000000274:head v 38'260, mlcod=37'256)", 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5113787900000002, 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001578716, 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056544+0000", 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5969999999999997e-06 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056558+0000", 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3900000000000001e-05 2024-09-13T05:02:36.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056564+0000", 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-06 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056627+0000", 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2762999999999995e-05 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056649+0000", 2024-09-13T05:02:36.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2086000000000002e-05 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.058077+0000", 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014276460000000001 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.058117+0000", 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0014e-05 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.2941857259999998, 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:36.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:02:36.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:36.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:36.660 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:36.662 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:36.663 INFO:teuthology.orchestra.run.smithi053.stdout:2878 write 0xd655051 thru 0xd659f17 (0x4ec7 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2879 write 0x73451e4 thru 0x7349635 (0x4452 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2880 punch from 0x52caf94 to 0x52cd748, (0x27b4 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2882 write 0xe2a54f0 thru 0xe2b06fd (0xb20e bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2884 punch from 0x41668b1 to 0x416b9bb, (0x510a bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2886 read 0x2a1e758 thru 0x2a250a8 (0x6951 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2887 writesame 0x225b178 thru 0x2267d48 data_size 0x1(0xcbd1 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2890 punch from 0xb82dbc1 to 0xb83373f, (0x5b7e bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2892 write 0xcc2bb77 thru 0xcc342cb (0x8755 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2893 read 0x67b10b4 thru 0x67b8333 (0x7280 bytes) 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2894 trunc from 0xe2b06fe to 0xf23e86 2024-09-13T05:02:36.664 INFO:teuthology.orchestra.run.smithi053.stdout:2895 punch from 0xdfaa5b to 0xe06ecc, (0xc471 bytes) 2024-09-13T05:02:36.665 INFO:teuthology.orchestra.run.smithi053.stdout:2896 writesame 0x1c58db3 thru 0x1c651d1 data_size 0x1(0xc41f bytes) 2024-09-13T05:02:36.665 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:36.661+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:36.674 INFO:teuthology.orchestra.run.smithi053.stdout:2897 writesame 0xbbfb596 thru 0xbbfca21 data_size 0xa(0x148c bytes) 2024-09-13T05:02:36.674 INFO:teuthology.orchestra.run.smithi053.stdout:2898 punch from 0x8ce87cd to 0x8cf0574, (0x7da7 bytes) 2024-09-13T05:02:36.674 INFO:teuthology.orchestra.run.smithi053.stdout:2899 read 0x9bd3371 thru 0x9bdbe50 (0x8ae0 bytes) 2024-09-13T05:02:36.674 INFO:teuthology.orchestra.run.smithi053.stdout:2900 write 0x6fbd273 thru 0x6fc4638 (0x73c6 bytes) 2024-09-13T05:02:36.677 INFO:teuthology.orchestra.run.smithi053.stdout:2901 write 0x51e1e8e thru 0x51ed6d2 (0xb845 bytes) 2024-09-13T05:02:36.680 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.680 INFO:teuthology.orchestra.run.smithi053.stdout:2903 punch from 0x59d3867 to 0x59e2dd5, (0xf56e bytes) 2024-09-13T05:02:36.687 INFO:teuthology.orchestra.run.smithi053.stdout:2904 writesame 0xcdf1588 thru 0xcdff935 data_size 0x2(0xe3ae bytes) 2024-09-13T05:02:36.687 INFO:teuthology.orchestra.run.smithi053.stdout:2905 read 0x2ca44f8 thru 0x2cb3446 (0xef4f bytes) 2024-09-13T05:02:36.687 INFO:teuthology.orchestra.run.smithi053.stdout:2906 writesame 0x294bfeb thru 0x2954b12 data_size 0x7a(0x8b28 bytes) 2024-09-13T05:02:36.689 INFO:teuthology.orchestra.run.smithi053.stdout:2907 write 0x8d7f9fe thru 0x8d87490 (0x7a93 bytes) 2024-09-13T05:02:36.692 INFO:teuthology.orchestra.run.smithi053.stdout:2910 write 0x40afeba thru 0x40ba410 (0xa557 bytes) 2024-09-13T05:02:36.693 INFO:teuthology.orchestra.run.smithi053.stdout:2911 punch from 0xb7b1e43 to 0xb7b363a, (0x17f7 bytes) 2024-09-13T05:02:36.694 INFO:teuthology.orchestra.run.smithi053.stdout:2912 read 0xa1d749b thru 0xa1d917c (0x1ce2 bytes) 2024-09-13T05:02:36.695 INFO:teuthology.orchestra.run.smithi053.stdout:2913 trunc from 0xcdff936 to 0xafe0f2c 2024-09-13T05:02:36.707 INFO:teuthology.orchestra.run.smithi053.stdout:2914 write 0xb1e6ae4 thru 0xb1e83df (0x18fc bytes) 2024-09-13T05:02:36.709 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.710 INFO:teuthology.orchestra.run.smithi053.stdout:2919 read 0x236bbd4 thru 0x237b542 (0xf96f bytes) 2024-09-13T05:02:36.710 INFO:teuthology.orchestra.run.smithi053.stdout:2921 punch from 0x8d4fdf to 0x8e0fea, (0xc00b bytes) 2024-09-13T05:02:36.715 INFO:teuthology.orchestra.run.smithi053.stdout:2922 write 0xb376f7e thru 0xb3852de (0xe361 bytes) 2024-09-13T05:02:36.719 INFO:teuthology.orchestra.run.smithi053.stdout:2923 read 0x78d96c8 thru 0x78ddeef (0x4828 bytes) 2024-09-13T05:02:36.719 INFO:teuthology.orchestra.run.smithi053.stdout:2925 read 0x689f784 thru 0x68af2c6 (0xfb43 bytes) 2024-09-13T05:02:36.719 INFO:teuthology.orchestra.run.smithi053.stdout:2926 write 0xaf16d97 thru 0xaf1e969 (0x7bd3 bytes) 2024-09-13T05:02:36.720 INFO:teuthology.orchestra.run.smithi053.stdout:2930 read 0x7591d75 thru 0x75938b1 (0x1b3d bytes) 2024-09-13T05:02:36.721 INFO:teuthology.orchestra.run.smithi053.stdout:2931 write 0x82d9852 thru 0x82e6506 (0xccb5 bytes) 2024-09-13T05:02:36.730 INFO:teuthology.orchestra.run.smithi053.stdout:2933 write 0xdcf30fa thru 0xdd020cd (0xefd4 bytes) 2024-09-13T05:02:36.734 INFO:teuthology.orchestra.run.smithi053.stdout:2934 punch from 0xafe2a8f to 0xafec25d, (0x97ce bytes) 2024-09-13T05:02:36.734 INFO:teuthology.orchestra.run.smithi053.stdout:2935 read 0xd1e6fdc thru 0xd1ec018 (0x503d bytes) 2024-09-13T05:02:36.734 INFO:teuthology.orchestra.run.smithi053.stdout:2937 write 0xb712ff8 thru 0xb715dac (0x2db5 bytes) 2024-09-13T05:02:36.737 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.737 INFO:teuthology.orchestra.run.smithi053.stdout:2939 write 0xa78f823 thru 0xa796142 (0x6920 bytes) 2024-09-13T05:02:36.739 INFO:teuthology.orchestra.run.smithi053.stdout:2941 read 0x46eaedd thru 0x46f00ec (0x5210 bytes) 2024-09-13T05:02:36.753 INFO:teuthology.orchestra.run.smithi053.stdout:2942 trunc from 0xdd020ce to 0x290a4e5 2024-09-13T05:02:36.763 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:36.762+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:36.767 INFO:teuthology.orchestra.run.smithi053.stdout:2944 writesame 0x6c032d thru 0x6c23b6 data_size 0x31(0x208a bytes) 2024-09-13T05:02:36.768 INFO:teuthology.orchestra.run.smithi053.stdout:2945 read 0xb53d44 thru 0xb5f567 (0xb824 bytes) 2024-09-13T05:02:36.769 INFO:teuthology.orchestra.run.smithi053.stdout:2946 read 0x236b458 thru 0x2372dfd (0x79a6 bytes) 2024-09-13T05:02:36.769 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:36.769 INFO:teuthology.orchestra.run.smithi053.stdout:2949 clone 17 order 21 su 8192 sc 7 2024-09-13T05:02:36.791 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:36.791 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:36.791 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:36.791 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:36.800 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:36.884 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:36.884+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:36.947 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:36.947 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:36.947 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:36.947 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:36.947 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:36.956 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:36.997 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:36.996+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:37.036 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone16 from 0x290a4e5 (overlap 0xf23e86) to 0x183973 2024-09-13T05:02:37.065 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:37.065+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:37.088 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-13T05:02:37.145 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.625206241000001, 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:02:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.625034935999999, 2024-09-13T05:02:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:02:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:1992 3.8 e25/19 ondisk, result = 0)", 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.618075203, 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016933153999999999, 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525025+0000", 2024-09-13T05:02:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525028+0000", 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1089999999999998e-06 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525038+0000", 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0042e-05 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.525042+0000", 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.51e-06 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541946+0000", 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.016903400999999998 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541951+0000", 2024-09-13T05:02:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7749999999999998e-06 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541958+0000", 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9489999999999999e-06 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:30249 3.b 3:d339a10a:::rbd_data.117c7aa26b99.0000000000000015:d [read 4087808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.188245+0000", 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.9548553949999992, 2024-09-13T05:02:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097714029999999997, 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 30249 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188245+0000", 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188245+0000", 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188245+0000", 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188247+0000", 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3140000000000002e-06 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188253+0000", 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2609999999999999e-06 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188259+0000", 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0900000000000001e-06 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.197830+0000", 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095702489999999994 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.197938+0000", 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000108731 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.198016+0000", 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8116999999999998e-05 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:30251 3.b 3:d339a10a:::rbd_data.117c7aa26b99.0000000000000015:d [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.188264+0000", 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.9548364429999996, 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098877989999999992, 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 30251 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188264+0000", 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188264+0000", 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188264+0000", 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188266+0000", 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3769999999999999e-06 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188272+0000", 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7830000000000004e-06 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.188277+0000", 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1070000000000004e-06 2024-09-13T05:02:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.198044+0000", 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097669799999999998 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.198110+0000", 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5969000000000005e-05 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.198152+0000", 2024-09-13T05:02:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1921000000000001e-05 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:31362 3.b 3:d339a10a:::rbd_data.117c7aa26b99.0000000000000015:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.252977+0000", 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.8901234799999997, 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097139229999999993, 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 31362 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.252977+0000", 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.252977+0000", 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.252976+0000", 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.252980+0000", 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4520000000000002e-06 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.252988+0000", 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6899999999999998e-06 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.252995+0000", 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3720000000000001e-06 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262565+0000", 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0095700990000000003 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262635+0000", 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0043999999999995e-05 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262691+0000", 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5832999999999999e-05 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:31364 3.b 3:d339a10a:::rbd_data.117c7aa26b99.0000000000000015:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.253023+0000", 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.8900776280000002, 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098472270000000001, 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 31364 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253023+0000", 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253023+0000", 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253022+0000", 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253026+0000", 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1309999999999999e-06 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253035+0000", 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7490000000000004e-06 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253041+0000", 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1659999999999999e-06 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262723+0000", 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096819620000000006 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262803+0000", 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0356000000000001e-05 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262870+0000", 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6626999999999998e-05 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:31366 3.b 3:d339a10a:::rbd_data.117c7aa26b99.0000000000000015:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.253213+0000", 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.8898874200000009, 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0097929599999999999, 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 31366 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253213+0000", 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253213+0000", 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253212+0000", 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253218+0000", 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7139999999999998e-06 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253231+0000", 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3339999999999999e-05 2024-09-13T05:02:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.253239+0000", 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6350000000000006e-06 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262893+0000", 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.009654246 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.262965+0000", 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2354999999999994e-05 2024-09-13T05:02:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.263006+0000", 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0629e-05 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:40342 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.933906+0000", 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2091940299999999, 2024-09-13T05:02:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.014226436, 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 40342 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.933906+0000", 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.933906+0000", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.933906+0000", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.933909+0000", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2480000000000001e-06 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.933915+0000", 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1129999999999997e-06 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.933920+0000", 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5759999999999999e-06 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.941464+0000", 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075445720000000003 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.941574+0000", 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000109283 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.941964+0000", 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039031899999999997 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.943352+0000", 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013878790000000001 2024-09-13T05:02:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.948072+0000", 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0047201129999999997 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.948107+0000", 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4863999999999997e-05 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.948133+0000", 2024-09-13T05:02:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5879000000000001e-05 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41033 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.060619+0000", 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0824814480000002, 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011455884, 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41033 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060619+0000", 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060619+0000", 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060619+0000", 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060621+0000", 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8310000000000001e-06 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060630+0000", 2024-09-13T05:02:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2670000000000006e-06 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060635+0000", 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5570000000000003e-06 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.068725+0000", 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0080897460000000001 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.068892+0000", 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00016656599999999999 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.069240+0000", 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034865299999999998 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.071932+0000", 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026915849999999998 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.071994+0000", 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2048e-05 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072040+0000", 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6288999999999997e-05 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072075+0000", 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4705000000000001e-05 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.195 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41034 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.060640+0000", 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0824606330000002, 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011520359000000001, 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41034 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.196 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060640+0000", 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060640+0000", 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060639+0000", 2024-09-13T05:02:37.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060650+0000", 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.045e-05 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060657+0000", 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9090000000000003e-06 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060662+0000", 2024-09-13T05:02:37.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6930000000000004e-06 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.069923+0000", 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0092606549999999996 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.069990+0000", 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6546999999999998e-05 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.070331+0000", 2024-09-13T05:02:37.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034182100000000002 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.071982+0000", 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016503659999999999 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072087+0000", 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000104957 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072127+0000", 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0309000000000003e-05 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072160+0000", 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3030000000000001e-05 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.201 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41035 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.060667+0000", 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0824333099999999, 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013699263, 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41035 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060667+0000", 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060667+0000", 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060667+0000", 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060669+0000", 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060676+0000", 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1870000000000002e-06 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060681+0000", 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8849999999999998e-06 2024-09-13T05:02:37.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.070943+0000", 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010262927999999999 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.071030+0000", 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6767000000000005e-05 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.071411+0000", 2024-09-13T05:02:37.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038065499999999998 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074226+0000", 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028151199999999999 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074280+0000", 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4314999999999998e-05 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074330+0000", 2024-09-13T05:02:37.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9867999999999998e-05 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074366+0000", 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6137999999999999e-05 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41036 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.060685+0000", 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0824157579999998, 2024-09-13T05:02:37.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013844400999999999, 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41036 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060685+0000", 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060685+0000", 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060684+0000", 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060687+0000", 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6699999999999998e-06 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060700+0000", 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2721000000000001e-05 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060706+0000", 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8619999999999999e-06 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072216+0000", 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011510805000000001 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072300+0000", 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3647999999999996e-05 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.072679+0000", 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00037940700000000001 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074425+0000", 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001745845 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074435+0000", 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9259999999999995e-06 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074481+0000", 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6192000000000001e-05 2024-09-13T05:02:37.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074529+0000", 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7679999999999998e-05 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41037 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.060711+0000", 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0823896670000002, 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015080705, 2024-09-13T05:02:37.213 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41037 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060711+0000", 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060711+0000", 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060710+0000", 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060713+0000", 2024-09-13T05:02:37.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060720+0000", 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060725+0000", 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2220000000000001e-06 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.073232+0000", 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.012507035 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.073316+0000", 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4579000000000005e-05 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.073698+0000", 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000381889 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.075702+0000", 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002003197 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.075709+0000", 2024-09-13T05:02:37.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9189999999999999e-06 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.075756+0000", 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6433000000000001e-05 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.075791+0000", 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5528000000000003e-05 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.219 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41038 3.2 3:47fd57f5:::rbd_object_map.117c4a06f7bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.060729+0000", 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.0823711219999996, 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015724017999999999, 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.220 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41038 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060729+0000", 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060729+0000", 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060729+0000", 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060731+0000", 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5170000000000002e-06 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060738+0000", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2720000000000003e-06 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.060743+0000", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0170000000000004e-06 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074566+0000", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.013823548 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074627+0000", 2024-09-13T05:02:37.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0408999999999997e-05 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.074986+0000", 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000359623 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.076122+0000", 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0011354920000000001 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.076395+0000", 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027293 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.076422+0000", 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7492999999999999e-05 2024-09-13T05:02:37.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.076453+0000", 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1096000000000002e-05 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41187 3.2 3:432f5011:::rbd_data.117c9ee8ea0.0000000000000000:10 [sparse-read 2031616~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.156870+0000", 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.9862307729999999, 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010077689000000001, 2024-09-13T05:02:37.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41187 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.156870+0000", 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.156870+0000", 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.156869+0000", 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.156871+0000", 2024-09-13T05:02:37.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.176e-06 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.156877+0000", 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7699999999999998e-06 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.156882+0000", 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.561e-06 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.166576+0000", 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0096940210000000006 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.166656+0000", 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9937000000000004e-05 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.166947+0000", 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00029151600000000002 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41775 3.b 3:d7fb6f72:::rbd_object_map.117cfc53257b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.328466+0000", 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8146344230000002, 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098918440000000003, 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41775 2024-09-13T05:02:37.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328466+0000", 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328466+0000", 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328466+0000", 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328469+0000", 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.269e-06 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328474+0000", 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.485e-06 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328479+0000", 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4190000000000004e-06 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.335261+0000", 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0067819639999999997 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.335359+0000", 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8677000000000004e-05 2024-09-13T05:02:37.234 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.335705+0000", 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034571500000000002 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.337365+0000", 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016598959999999999 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338271+0000", 2024-09-13T05:02:37.235 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00090565999999999997 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338320+0000", 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8914e-05 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338358+0000", 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8294000000000003e-05 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.236 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41776 3.b 3:d7fb6f72:::rbd_object_map.117cfc53257b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.328586+0000", 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.814514306, 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0098936690000000008, 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41776 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.237 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328586+0000", 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328586+0000", 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328585+0000", 2024-09-13T05:02:37.238 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328591+0000", 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1940000000000003e-06 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328604+0000", 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3039e-05 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.239 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328612+0000", 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.336119+0000", 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0075070900000000001 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.336231+0000", 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011209699999999999 2024-09-13T05:02:37.240 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.336616+0000", 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00038485499999999997 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338369+0000", 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001752806 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.241 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338414+0000", 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4957999999999999e-05 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338451+0000", 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6844000000000002e-05 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338480+0000", 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9047000000000002e-05 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.242 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41777 3.b 3:d7fb6f72:::rbd_object_map.117cfc53257b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.328812+0000", 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8142884960000001, 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010797667, 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41777 2024-09-13T05:02:37.243 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328812+0000", 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328812+0000", 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.244 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328811+0000", 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328817+0000", 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328827+0000", 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0259e-05 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.245 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.328834+0000", 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9059999999999996e-06 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.337389+0000", 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0085547550000000007 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.337469+0000", 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0259999999999994e-05 2024-09-13T05:02:37.246 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.337834+0000", 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00036458999999999998 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.339340+0000", 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001506212 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.339521+0000", 2024-09-13T05:02:37.247 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00018040800000000001 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.339574+0000", 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3782000000000001e-05 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.339610+0000", 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5194999999999998e-05 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.248 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:41778 3.b 3:d7fb6f72:::rbd_object_map.117cfc53257b:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e38)", 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.329590+0000", 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.8135099119999998, 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01059965, 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 41778 2024-09-13T05:02:37.249 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.329590+0000", 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.329590+0000", 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.329589+0000", 2024-09-13T05:02:37.250 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.329596+0000", 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1430000000000003e-06 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.329610+0000", 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4368e-05 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.329618+0000", 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7060000000000004e-06 2024-09-13T05:02:37.251 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338514+0000", 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0088960689999999995 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338588+0000", 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4727999999999993e-05 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 0", 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.338930+0000", 2024-09-13T05:02:37.252 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034171100000000002 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.340004+0000", 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00107412 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.340064+0000", 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0183999999999998e-05 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.253 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.340112+0000", 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7648999999999997e-05 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.340190+0000", 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8041999999999996e-05 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:37.254 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:37.256 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:37.256 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:37.255+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:37.326 INFO:teuthology.orchestra.run.smithi053.stdout:2950 write 0x42db625 thru 0x42e20e3 (0x6abf bytes) 2024-09-13T05:02:37.336 INFO:teuthology.orchestra.run.smithi053.stdout:2951 write 0x4538fe8 thru 0x453ffb5 (0x6fce bytes) 2024-09-13T05:02:37.339 INFO:teuthology.orchestra.run.smithi053.stdout:2952 read 0x31fa824 thru 0x3203089 (0x8866 bytes) 2024-09-13T05:02:37.347 INFO:teuthology.orchestra.run.smithi053.stdout:2953 trunc from 0x453ffb6 to 0xb308103 2024-09-13T05:02:37.353 INFO:teuthology.orchestra.run.smithi053.stdout:2954 punch from 0x74d6013 to 0x74dabc8, (0x4bb5 bytes) 2024-09-13T05:02:37.353 INFO:teuthology.orchestra.run.smithi053.stdout:2955 write 0xc24e9d thru 0xc339e7 (0xeb4b bytes) 2024-09-13T05:02:37.360 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:37.360+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:37.373 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:37.373 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:37.373 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:37.374 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:37.383 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:37.468 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:37.468+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:37.526 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:37.527 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:37.527 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:37.527 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:37.527 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:37.536 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:37.577 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:37.576+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:37.601 INFO:teuthology.orchestra.run.smithi053.stdout:2956 read 0x4feba22 thru 0x4ff052a (0x4b09 bytes) 2024-09-13T05:02:37.602 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:37.602 INFO:teuthology.orchestra.run.smithi053.stdout:2958 write 0x73b875 thru 0x741b48 (0x62d4 bytes) 2024-09-13T05:02:37.604 INFO:teuthology.orchestra.run.smithi053.stdout:2959 punch from 0x5794773 to 0x579dc83, (0x9510 bytes) 2024-09-13T05:02:37.604 INFO:teuthology.orchestra.run.smithi053.stdout:2960 write 0xab92bff thru 0xab951b6 (0x25b8 bytes) 2024-09-13T05:02:37.606 INFO:teuthology.orchestra.run.smithi053.stdout:2961 read 0x714bda thru 0x714f36 (0x35d bytes) 2024-09-13T05:02:37.609 INFO:teuthology.orchestra.run.smithi053.stdout:2962 trunc from 0xb308103 to 0xcb27df7 2024-09-13T05:02:37.620 INFO:teuthology.orchestra.run.smithi053.stdout:2963 write 0xd9848b3 thru 0xd98621f (0x196d bytes) 2024-09-13T05:02:37.623 INFO:teuthology.orchestra.run.smithi053.stdout:2964 write 0x2165e9d thru 0x2167400 (0x1564 bytes) 2024-09-13T05:02:37.658 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:37.658+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:37.659 INFO:teuthology.orchestra.run.smithi053.stdout:2965 trunc from 0xd986220 to 0x180402e 2024-09-13T05:02:37.720 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:37.720 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475640729, 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.721 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:02:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:02:37.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:02:37.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475624474, 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:37.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:02:37.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475607385, 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:02:37.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:02:37.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475590518000001, 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:02:37.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.740 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.47554122, 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:37.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:02:37.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475517951, 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.745 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:02:37.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.749 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475502217000001, 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:02:37.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:02:37.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475479237, 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.754 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:02:37.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:37.758 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475430170999999, 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:02:37.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475308856, 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.763 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:02:37.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475253642, 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475233353, 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.772 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.776 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475216172, 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:02:37.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475128661999999, 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.781 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:02:37.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:02:37.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475109414, 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.786 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:02:37.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:37.790 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475048299000001, 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:37.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:02:37.794 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475031124000001, 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.795 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:02:37.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:37.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.475014714, 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:02:37.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.803 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.47499373, 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.804 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:02:37.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:02:37.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.474976306, 2024-09-13T05:02:37.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:02:37.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:02:37.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:37.813 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:37.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:37.816 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:37.816 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:37.813+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:2966 write 0x81d7680 thru 0x81e6945 (0xf2c6 bytes) 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:2967 writesame 0xb2090a4 thru 0xb216e8e data_size 0x1d(0xddeb bytes) 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:2968 write 0x4a08e44 thru 0x4a16a71 (0xdc2e bytes) 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:2969 read 0x5394ceb thru 0x539e317 (0x962d bytes) 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:2972 trunc from 0xb216e8f to 0x979c2c5 2024-09-13T05:02:37.816 INFO:teuthology.orchestra.run.smithi053.stdout:2973 write 0xc65852f thru 0xc65b244 (0x2d16 bytes) 2024-09-13T05:02:37.818 INFO:teuthology.orchestra.run.smithi053.stdout:2974 trunc from 0xc65b245 to 0x78fb963 2024-09-13T05:02:37.832 INFO:teuthology.orchestra.run.smithi053.stdout:2976 trunc from 0x78fb963 to 0x53d8168 2024-09-13T05:02:37.839 INFO:teuthology.orchestra.run.smithi053.stdout:2977 clone 18 order 25 su 2097152 sc 10 2024-09-13T05:02:37.915 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:37.914+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:37.942 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:37.942 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:37.959 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:37.959 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:37.960 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:37.960 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:37.960 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:37.960 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:37.969 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:37.975 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:37.975 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:37.990 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:02:37.995 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:37.995 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:38.013 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:38.013 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:38.016 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:38.016+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:38.033 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:38.034 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:38.046 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone17 from 0x53d8168 (overlap 0x180402e) to 0x133668b 2024-09-13T05:02:38.053 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:38.053 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:38.074 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:38.074 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:38.075 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.076 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.085 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:38.126 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-13T05:02:38.127 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:38.126+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:38.130 INFO:teuthology.orchestra.run.smithi053.stdout:2979 punch from 0x4ccab39 to 0x4cd886f, (0xdd36 bytes) 2024-09-13T05:02:38.147 INFO:teuthology.orchestra.run.smithi053.stdout:2982 writesame 0xcb9279b thru 0xcba0cac data_size 0x6d(0xe512 bytes) 2024-09-13T05:02:38.156 INFO:teuthology.orchestra.run.smithi053.stdout:2983 read 0x68d5a1b thru 0x68d9aad (0x4093 bytes) 2024-09-13T05:02:38.156 INFO:teuthology.orchestra.run.smithi053.stdout:2985 punch from 0x86e1735 to 0x86efc36, (0xe501 bytes) 2024-09-13T05:02:38.156 INFO:teuthology.orchestra.run.smithi053.stdout:2986 write 0x1980efa thru 0x198cf49 (0xc050 bytes) 2024-09-13T05:02:38.162 INFO:teuthology.orchestra.run.smithi053.stdout:2987 writesame 0xb727803 thru 0xb732ebe data_size 0xa(0xb6bc bytes) 2024-09-13T05:02:38.170 INFO:teuthology.orchestra.run.smithi053.stdout:2989 trunc from 0xcba0cad to 0x753e04c 2024-09-13T05:02:38.196 INFO:teuthology.orchestra.run.smithi053.stdout:2991 writesame 0x17e3476 thru 0x17e9531 data_size 0x52(0x60bc bytes) 2024-09-13T05:02:38.197 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.197 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.197 INFO:teuthology.orchestra.run.smithi053.stdout:2992 write 0x2dcc44 thru 0x2e56f5 (0x8ab2 bytes) 2024-09-13T05:02:38.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:38.207 INFO:teuthology.orchestra.run.smithi053.stdout:2994 trunc from 0x753e04c to 0xdce2ce1 2024-09-13T05:02:38.212 INFO:teuthology.orchestra.run.smithi053.stdout:2995 read 0x6a7880f thru 0x6a8357b (0xad6d bytes) 2024-09-13T05:02:38.213 INFO:teuthology.orchestra.run.smithi053.stdout:2996 write 0xc918907 thru 0xc923db9 (0xb4b3 bytes) 2024-09-13T05:02:38.215 INFO:teuthology.orchestra.run.smithi053.stdout:2997 write 0x41ba814 thru 0x41bb7d7 (0xfc4 bytes) 2024-09-13T05:02:38.216 INFO:teuthology.orchestra.run.smithi053.stdout:2998 write 0x2078fee thru 0x207a8b2 (0x18c5 bytes) 2024-09-13T05:02:38.218 INFO:teuthology.orchestra.run.smithi053.stdout:3000 read 0x69aeae5 thru 0x69b446e (0x598a bytes) 2024-09-13T05:02:38.218 INFO:teuthology.orchestra.run.smithi053.stdout:3002 write 0x491224f thru 0x4920bd1 (0xe983 bytes) 2024-09-13T05:02:38.225 INFO:teuthology.orchestra.run.smithi053.stdout:3003 trunc from 0xdce2ce1 to 0x98486cf 2024-09-13T05:02:38.233 INFO:teuthology.orchestra.run.smithi053.stdout:3004 punch from 0xb4b656 to 0xb59ab5, (0xe45f bytes) 2024-09-13T05:02:38.233 INFO:teuthology.orchestra.run.smithi053.stdout:3005 write 0x48c318b thru 0x48c6c39 (0x3aaf bytes) 2024-09-13T05:02:38.236 INFO:teuthology.orchestra.run.smithi053.stdout:3006 trunc from 0x98486cf to 0xb4de7ef 2024-09-13T05:02:38.241 INFO:teuthology.orchestra.run.smithi053.stdout:3007 write 0x186246c thru 0x186d159 (0xacee bytes) 2024-09-13T05:02:38.243 INFO:teuthology.orchestra.run.smithi053.stdout:3012 write 0x9c24857 thru 0x9c2787f (0x3029 bytes) 2024-09-13T05:02:38.245 INFO:teuthology.orchestra.run.smithi053.stdout:3013 punch from 0x68e0660 to 0x68eac6a, (0xa60a bytes) 2024-09-13T05:02:38.245 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:38.246 INFO:teuthology.orchestra.run.smithi053.stdout:3015 writesame 0x6b1ac67 thru 0x6b286e5 data_size 0x71(0xda7f bytes) 2024-09-13T05:02:38.246 INFO:teuthology.orchestra.run.smithi053.stdout:3018 punch from 0x6cd5793 to 0x6cdee1f, (0x968c bytes) 2024-09-13T05:02:38.251 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:38.250+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:38.252 INFO:teuthology.orchestra.run.smithi053.stdout:3019 trunc from 0xb4de7ef to 0x71b92ca 2024-09-13T05:02:38.258 INFO:teuthology.orchestra.run.smithi053.stdout:3023 punch from 0x3af934a to 0x3b01d62, (0x8a18 bytes) 2024-09-13T05:02:38.258 INFO:teuthology.orchestra.run.smithi053.stdout:3024 writesame 0x62e4782 thru 0x62eae02 data_size 0x1(0x6681 bytes) 2024-09-13T05:02:38.273 INFO:teuthology.orchestra.run.smithi053.stdout:3025 read 0x5627636 thru 0x562a2b1 (0x2c7c bytes) 2024-09-13T05:02:38.280 INFO:teuthology.orchestra.run.smithi053.stdout:3026 trunc from 0x71b92ca to 0x18a66c7 2024-09-13T05:02:38.287 INFO:teuthology.orchestra.run.smithi053.stdout:3027 punch from 0x16cbd49 to 0x16d7731, (0xb9e8 bytes) 2024-09-13T05:02:38.287 INFO:teuthology.orchestra.run.smithi053.stdout:3028 punch from 0x89883a to 0x8a1939, (0x90ff bytes) 2024-09-13T05:02:38.288 INFO:teuthology.orchestra.run.smithi053.stdout:3029 read 0x1790d7d thru 0x179c6a3 (0xb927 bytes) 2024-09-13T05:02:38.300 INFO:teuthology.orchestra.run.smithi053.stdout:3030 trunc from 0x18a66c7 to 0xcf43b01 2024-09-13T05:02:38.306 INFO:teuthology.orchestra.run.smithi053.stdout:3031 writesame 0x2a3d94c thru 0x2a46308 data_size 0x1(0x89bd bytes) 2024-09-13T05:02:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.318 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:38.317+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:38.322 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:38.329 INFO:teuthology.orchestra.run.smithi053.stdout:3034 read 0x7b2e9ee thru 0x7b396b3 (0xacc6 bytes) 2024-09-13T05:02:38.330 INFO:teuthology.orchestra.run.smithi053.stdout:3037 write 0x1b63c63 thru 0x1b68fd5 (0x5373 bytes) 2024-09-13T05:02:38.331 INFO:teuthology.orchestra.run.smithi053.stdout:3038 write 0xa8a9ff7 thru 0xa8b3574 (0x957e bytes) 2024-09-13T05:02:38.333 INFO:teuthology.orchestra.run.smithi053.stdout:3039 write 0x4859935 thru 0x48605eb (0x6cb7 bytes) 2024-09-13T05:02:38.342 INFO:teuthology.orchestra.run.smithi053.stdout:3041 write 0xe9ba4e7 thru 0xe9ba649 (0x163 bytes) 2024-09-13T05:02:38.343 INFO:teuthology.orchestra.run.smithi053.stdout:3042 punch from 0xc302b2d to 0xc3073eb, (0x48be bytes) 2024-09-13T05:02:38.350 INFO:teuthology.orchestra.run.smithi053.stdout:3043 trunc from 0xe9ba64a to 0x8b98b6b 2024-09-13T05:02:38.357 INFO:teuthology.orchestra.run.smithi053.stdout:3044 read 0x9bcaa9 thru 0x9c432f (0x7887 bytes) 2024-09-13T05:02:38.357 INFO:teuthology.orchestra.run.smithi053.stdout:3045 write 0x4ee14fc thru 0x4ee7568 (0x606d bytes) 2024-09-13T05:02:38.358 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:38.359 INFO:teuthology.orchestra.run.smithi053.stdout:3047 read 0x6fced8 thru 0x7056fb (0x8824 bytes) 2024-09-13T05:02:38.359 INFO:teuthology.orchestra.run.smithi053.stdout:3048 read 0x2e3b498 thru 0x2e48f3b (0xdaa4 bytes) 2024-09-13T05:02:38.359 INFO:teuthology.orchestra.run.smithi053.stdout:3049 read 0x374b573 thru 0x3756bdf (0xb66d bytes) 2024-09-13T05:02:38.367 INFO:teuthology.orchestra.run.smithi053.stdout:3051 writesame 0xb084c63 thru 0xb08946e data_size 0x3a(0x480c bytes) 2024-09-13T05:02:38.368 INFO:teuthology.orchestra.run.smithi053.stdout:3054 read 0x884c461 thru 0x8854ad0 (0x8670 bytes) 2024-09-13T05:02:38.369 INFO:teuthology.orchestra.run.smithi053.stdout:3056 read 0x238010f thru 0x23824e5 (0x23d7 bytes) 2024-09-13T05:02:38.371 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:38.371 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:38.371 INFO:teuthology.orchestra.run.smithi053.stdout:3060 read 0xa9da00c thru 0xa9dc64b (0x2640 bytes) 2024-09-13T05:02:38.371 INFO:teuthology.orchestra.run.smithi053.stdout:3061 read 0x403503 thru 0x408bcf (0x56cd bytes) 2024-09-13T05:02:38.372 INFO:teuthology.orchestra.run.smithi053.stdout:3062 read 0x8f8cac thru 0x8fdb9a (0x4eef bytes) 2024-09-13T05:02:38.372 INFO:teuthology.orchestra.run.smithi053.stdout:3063 write 0x16aaf19 thru 0x16b15c6 (0x66ae bytes) 2024-09-13T05:02:38.376 INFO:teuthology.orchestra.run.smithi053.stdout:3067 writesame 0xb10a906 thru 0xb11716d data_size 0x79(0xc868 bytes) 2024-09-13T05:02:38.377 INFO:teuthology.orchestra.run.smithi053.stdout:3068 read 0x3a4506c thru 0x3a4a1fe (0x5193 bytes) 2024-09-13T05:02:38.377 INFO:teuthology.orchestra.run.smithi053.stdout:3069 write 0xa255376 thru 0xa25ec10 (0x989b bytes) 2024-09-13T05:02:38.380 INFO:teuthology.orchestra.run.smithi053.stdout:3071 trunc from 0xb11716e to 0xa516bc6 2024-09-13T05:02:38.387 INFO:teuthology.orchestra.run.smithi053.stdout:3074 write 0x5388ede thru 0x5396b3a (0xdc5d bytes) 2024-09-13T05:02:38.389 INFO:teuthology.orchestra.run.smithi053.stdout:3076 writesame 0x992d509 thru 0x9939aa0 data_size 0x4(0xc598 bytes) 2024-09-13T05:02:38.394 INFO:teuthology.orchestra.run.smithi053.stdout:3077 write 0xa37a774 thru 0xa37ce5d (0x26ea bytes) 2024-09-13T05:02:38.395 INFO:teuthology.orchestra.run.smithi053.stdout:3078 read 0x91ffe0b thru 0x920795c (0x7b52 bytes) 2024-09-13T05:02:38.395 INFO:teuthology.orchestra.run.smithi053.stdout:3079 read 0x3ab771c thru 0x3ab8248 (0xb2d bytes) 2024-09-13T05:02:38.396 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:38.396 INFO:teuthology.orchestra.run.smithi053.stdout:3082 read 0x4abdb6e thru 0x4ac9ef4 (0xc387 bytes) 2024-09-13T05:02:38.396 INFO:teuthology.orchestra.run.smithi053.stdout:3084 write 0x6c7082 thru 0x6d31fa (0xc179 bytes) 2024-09-13T05:02:38.398 INFO:teuthology.orchestra.run.smithi053.stdout:3085 read 0xa05003d thru 0xa054ea2 (0x4e66 bytes) 2024-09-13T05:02:38.398 INFO:teuthology.orchestra.run.smithi053.stdout:3086 write 0x115c413 thru 0x115f2ea (0x2ed8 bytes) 2024-09-13T05:02:38.400 INFO:teuthology.orchestra.run.smithi053.stdout:3087 write 0x6cca78d thru 0x6cce69f (0x3f13 bytes) 2024-09-13T05:02:38.407 INFO:teuthology.orchestra.run.smithi053.stdout:3089 write 0xaf19422 thru 0xaf25cab (0xc88a bytes) 2024-09-13T05:02:38.409 INFO:teuthology.orchestra.run.smithi053.stdout:3090 write 0xa69dbce thru 0xa6a1048 (0x347b bytes) 2024-09-13T05:02:38.410 INFO:teuthology.orchestra.run.smithi053.stdout:3092 write 0x57fe30a thru 0x58096f8 (0xb3ef bytes) 2024-09-13T05:02:38.412 INFO:teuthology.orchestra.run.smithi053.stdout:3093 write 0x99c6b0d thru 0x99c7e98 (0x138c bytes) 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:3094 read 0x2e20263 thru 0x2e2841e (0x81bc bytes) 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:3096 read 0x9b21806 thru 0x9b30db4 (0xf5af bytes) 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:3097 read 0x923d070 thru 0x924cdbe (0xfd4f bytes) 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:3100 punch from 0x75f182c to 0x75f2752, (0xf26 bytes) 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:3101 write 0x6b9c482 thru 0x6ba1c75 (0x57f4 bytes) 2024-09-13T05:02:38.415 INFO:teuthology.orchestra.run.smithi053.stdout:3102 clone 19 order 24 su 262144 sc 2 2024-09-13T05:02:38.419 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:38.418+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:38.465 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.466 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.474 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:38.519 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:38.518+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:38.575 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.575 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.583 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:38.625 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:38.624+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:38.721 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:38.720+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:38.725 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.726 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.734 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:38.822 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:38.821+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:38.875 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.876 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.884 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:38.923 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:38.922+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:38.985 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:02:38.986 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:38.987 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:38.996 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:39.043 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:39.042+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:39.060 INFO:teuthology.orchestra.run.smithi053.stdout:flattening image image_client.0-clone18 2024-09-13T05:02:39.121 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:39.121 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:39.126 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:39.124+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:39.131 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:39.197 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-13T05:02:39.226 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:39.225+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:39.232 INFO:teuthology.orchestra.run.smithi053.stdout:3106 trunc from 0xaf25cac to 0x666246e 2024-09-13T05:02:39.255 INFO:teuthology.orchestra.run.smithi053.stdout:3107 writesame 0x42705db thru 0x427e69f data_size 0x1(0xe0c5 bytes) 2024-09-13T05:02:39.287 INFO:teuthology.orchestra.run.smithi053.stdout:3108 write 0x283521d thru 0x283e19a (0x8f7e bytes) 2024-09-13T05:02:39.296 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:39.296 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:39.297 INFO:teuthology.orchestra.run.smithi053.stdout:3109 read 0x2900cd2 thru 0x290e43c (0xd76b bytes) 2024-09-13T05:02:39.297 INFO:teuthology.orchestra.run.smithi053.stdout:3110 read 0x51ed2a9 thru 0x51f3305 (0x605d bytes) 2024-09-13T05:02:39.297 INFO:teuthology.orchestra.run.smithi053.stdout:3111 read 0x23d9d48 thru 0x23e390c (0x9bc5 bytes) 2024-09-13T05:02:39.297 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.297 INFO:teuthology.orchestra.run.smithi053.stdout:3116 punch from 0x2b0fe55 to 0x2b16cd3, (0x6e7e bytes) 2024-09-13T05:02:39.305 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:39.309 INFO:teuthology.orchestra.run.smithi053.stdout:3119 write 0xdc37ed8 thru 0xdc440f9 (0xc222 bytes) 2024-09-13T05:02:39.325 INFO:teuthology.orchestra.run.smithi053.stdout:3120 trunc from 0xdc440fa to 0x2c54e30 2024-09-13T05:02:39.328 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:39.326+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:39.344 INFO:teuthology.orchestra.run.smithi053.stdout:3121 writesame 0x1f92725 thru 0x1f9ab3d data_size 0x1(0x8419 bytes) 2024-09-13T05:02:39.378 INFO:teuthology.orchestra.run.smithi053.stdout:3122 read 0x2ac8650 thru 0x2acf54c (0x6efd bytes) 2024-09-13T05:02:39.379 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.388 INFO:teuthology.orchestra.run.smithi053.stdout:3125 writesame 0x655a7bd thru 0x655bdc6 data_size 0x1f(0x160a bytes) 2024-09-13T05:02:39.388 INFO:teuthology.orchestra.run.smithi053.stdout:3127 read 0x5c0439e thru 0x5c0a1c8 (0x5e2b bytes) 2024-09-13T05:02:39.397 INFO:teuthology.orchestra.run.smithi053.stdout:3129 write 0xa05caed thru 0xa068f5d (0xc471 bytes) 2024-09-13T05:02:39.406 INFO:teuthology.orchestra.run.smithi053.stdout:3130 trunc from 0xa068f5e to 0x5132ff2 2024-09-13T05:02:39.421 INFO:teuthology.orchestra.run.smithi053.stdout:3132 write 0x7516207 thru 0x75196a5 (0x349f bytes) 2024-09-13T05:02:39.429 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:39.427+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:39.431 INFO:teuthology.orchestra.run.smithi053.stdout:3133 write 0x97aa5f4 thru 0x97b07f6 (0x6203 bytes) 2024-09-13T05:02:39.437 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:39.437 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:39.441 INFO:teuthology.orchestra.run.smithi053.stdout:3134 writesame 0xa7ba2e2 thru 0xa7c5013 data_size 0x7(0xad32 bytes) 2024-09-13T05:02:39.441 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.441 INFO:teuthology.orchestra.run.smithi053.stdout:3137 punch from 0xa6a31e9 to 0xa6b1283, (0xe09a bytes) 2024-09-13T05:02:39.441 INFO:teuthology.orchestra.run.smithi053.stdout:3138 write 0x745f47b thru 0x746352a (0x40b0 bytes) 2024-09-13T05:02:39.447 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:39.452 INFO:teuthology.orchestra.run.smithi053.stdout:3139 trunc from 0xa7c5014 to 0x3cde5ee 2024-09-13T05:02:39.465 INFO:teuthology.orchestra.run.smithi053.stdout:3143 write 0x2c698c7 thru 0x2c6c5f5 (0x2d2f bytes) 2024-09-13T05:02:39.466 INFO:teuthology.orchestra.run.smithi053.stdout:3144 punch from 0x260abff to 0x261895d, (0xdd5e bytes) 2024-09-13T05:02:39.466 INFO:teuthology.orchestra.run.smithi053.stdout:3145 read 0x1e754db thru 0x1e80c5c (0xb782 bytes) 2024-09-13T05:02:39.466 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.466 INFO:teuthology.orchestra.run.smithi053.stdout:3147 writesame 0x30ee112 thru 0x30f1546 data_size 0x1b(0x3435 bytes) 2024-09-13T05:02:39.467 INFO:teuthology.orchestra.run.smithi053.stdout:3148 read 0x2ce9d91 thru 0x2cf542d (0xb69d bytes) 2024-09-13T05:02:39.468 INFO:teuthology.orchestra.run.smithi053.stdout:3149 punch from 0x23e7b17 to 0x23e7e8d, (0x376 bytes) 2024-09-13T05:02:39.468 INFO:teuthology.orchestra.run.smithi053.stdout:3150 write 0x274aede thru 0x274cce8 (0x1e0b bytes) 2024-09-13T05:02:39.473 INFO:teuthology.orchestra.run.smithi053.stdout:3152 trunc from 0x3cde5ee to 0x70b9df0 2024-09-13T05:02:39.478 INFO:teuthology.orchestra.run.smithi053.stdout:3153 write 0x66e79de thru 0x66f058b (0x8bae bytes) 2024-09-13T05:02:39.482 INFO:teuthology.orchestra.run.smithi053.stdout:3154 punch from 0x21b86a3 to 0x21c3cb8, (0xb615 bytes) 2024-09-13T05:02:39.482 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.482 INFO:teuthology.orchestra.run.smithi053.stdout:3157 read 0x460b20 thru 0x46b6c7 (0xaba8 bytes) 2024-09-13T05:02:39.488 INFO:teuthology.orchestra.run.smithi053.stdout:3158 write 0x7eeb35a thru 0x7efae38 (0xfadf bytes) 2024-09-13T05:02:39.504 INFO:teuthology.orchestra.run.smithi053.stdout:3159 write 0xed5f73b thru 0xed68d0e (0x95d4 bytes) 2024-09-13T05:02:39.511 INFO:teuthology.orchestra.run.smithi053.stdout:3160 trunc from 0xed68d0f to 0xc53a596 2024-09-13T05:02:39.519 INFO:teuthology.orchestra.run.smithi053.stdout:3161 read 0x4e0ce3f thru 0x4e164c0 (0x9682 bytes) 2024-09-13T05:02:39.520 INFO:teuthology.orchestra.run.smithi053.stdout:3164 read 0x32f8814 thru 0x32f94d1 (0xcbe bytes) 2024-09-13T05:02:39.520 INFO:teuthology.orchestra.run.smithi053.stdout:3165 punch from 0x61b9003 to 0x61c239e, (0x939b bytes) 2024-09-13T05:02:39.526 INFO:teuthology.orchestra.run.smithi053.stdout:3166 write 0xd2ea752 thru 0xd2f03a2 (0x5c51 bytes) 2024-09-13T05:02:39.529 INFO:teuthology.orchestra.run.smithi053.stdout:3168 write 0x9686b05 thru 0x968e29f (0x779b bytes) 2024-09-13T05:02:39.529 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:39.528+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:39.533 INFO:teuthology.orchestra.run.smithi053.stdout:3169 punch from 0x9e64fa5 to 0x9e7452f, (0xf58a bytes) 2024-09-13T05:02:39.533 INFO:teuthology.orchestra.run.smithi053.stdout:3170 writesame 0x43a8350 thru 0x43b5bf6 data_size 0x1(0xd8a7 bytes) 2024-09-13T05:02:39.573 INFO:teuthology.orchestra.run.smithi053.stdout:3171 trunc from 0xd2f03a3 to 0x325d3f8 2024-09-13T05:02:39.583 INFO:teuthology.orchestra.run.smithi053.stdout:3172 punch from 0x31d3222 to 0x31e1ef0, (0xecce bytes) 2024-09-13T05:02:39.590 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:39.590 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:39.592 INFO:teuthology.orchestra.run.smithi053.stdout:3173 writesame 0x5eb866b thru 0x5eb9b22 data_size 0x34(0x14b8 bytes) 2024-09-13T05:02:39.592 INFO:teuthology.orchestra.run.smithi053.stdout:3174 writesame 0x551e38a thru 0x552de02 data_size 0x1(0xfa79 bytes) 2024-09-13T05:02:39.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:39.624 INFO:teuthology.orchestra.run.smithi053.stdout:3177 read 0xb5cebb thru 0xb6c175 (0xf2bb bytes) 2024-09-13T05:02:39.625 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.625 INFO:teuthology.orchestra.run.smithi053.stdout:3181 read 0x2772ed5 thru 0x2777afb (0x4c27 bytes) 2024-09-13T05:02:39.626 INFO:teuthology.orchestra.run.smithi053.stdout:3182 writesame 0x2ae3ccf thru 0x2aef501 data_size 0x1(0xb833 bytes) 2024-09-13T05:02:39.648 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.648 INFO:teuthology.orchestra.run.smithi053.stdout:3184 write 0x3de61a3 thru 0x3defafa (0x9958 bytes) 2024-09-13T05:02:39.648 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:39.646+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:39.649 INFO:teuthology.orchestra.run.smithi053.stdout:3185 read 0x49b97ff thru 0x49c505b (0xb85d bytes) 2024-09-13T05:02:39.662 INFO:teuthology.orchestra.run.smithi053.stdout:3186 writesame 0xc4d72f3 thru 0xc4d9877 data_size 0x11(0x2585 bytes) 2024-09-13T05:02:39.662 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.663 INFO:teuthology.orchestra.run.smithi053.stdout:3188 read 0xb935ba2 thru 0xb94015f (0xa5be bytes) 2024-09-13T05:02:39.663 INFO:teuthology.orchestra.run.smithi053.stdout:3190 read 0x5d15274 thru 0x5d2144c (0xc1d9 bytes) 2024-09-13T05:02:39.663 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:39.663 INFO:teuthology.orchestra.run.smithi053.stdout:3193 read 0xaf97e4b thru 0xafa6b87 (0xed3d bytes) 2024-09-13T05:02:39.663 INFO:teuthology.orchestra.run.smithi053.stdout:3195 clone 20 order 18 su 65536 sc 4 2024-09-13T05:02:39.732 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:39.731+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:39.732 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:39.732 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:39.741 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:39.832 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:39.831+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:39.887 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:39.896 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:39.935 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:39.933+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:39.986 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:39.986 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:39.987 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T05:02:39.987 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:02:39.987 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:02:39.987 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T05:02:40.036 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:40.034+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:40.043 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone19 from 0xc4d9878 (overlap 0x2c54e30) to 0x1a99550 2024-09-13T05:02:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:40.049 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:40.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:40.088 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:40.092 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:40.093 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "" 2024-09-13T05:02:40.093 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:40.136 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-13T05:02:40.137 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:40.135+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:40.187 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:40.187 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:40.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:40.237 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:40.235+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:40.244 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:40.245 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:40.263 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:40.264 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:40.282 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:40.283 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:40.298 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:40.299 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:40.315 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:40.315 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:3196 flatten 2024-09-13T05:02:40.332 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:40.332 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:40.338 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:40.336+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:40.364 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:40.365 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:40.374 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:40.374 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:9144 3.4 e28/19 3:21949514:::rbd_data.117c6c62fa22.0000000000000052:head v 28'73, mlcod=27'71)", 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.459746289999998, 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001561832, 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912415+0000", 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912419+0000", 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8749999999999999e-06 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912432+0000", 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2373e-05 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.912440+0000", 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2889999999999998e-06 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913219+0000", 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077888000000000002 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913240+0000", 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0672000000000001e-05 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913941+0000", 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00070157300000000004 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:17.913977+0000", 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5472000000000002e-05 2024-09-13T05:02:40.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.394097198000001, 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:02:40.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10094 3.4 e29/19 3:24263961:::rbd_data.117c6c62fa22.000000000000000b:head v 29'95, mlcod=28'92)", 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.391025789, 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001767984, 2024-09-13T05:02:40.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:40.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981152+0000", 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6986999999999998e-05 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981166+0000", 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4331000000000001e-05 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981177+0000", 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0858e-05 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981233+0000", 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5813000000000002e-05 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981253+0000", 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9707999999999998e-05 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982869+0000", 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001615932 2024-09-13T05:02:40.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982903+0000", 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4626999999999999e-05 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10141 3.4 e29/19 3:24508b78:::rbd_data.117c6c62fa22.000000000000003d:head v 29'101, mlcod=29'97)", 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:40.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.377283076000001, 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001501142, 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994878+0000", 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994877+0000", 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994885+0000", 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9869999999999999e-06 2024-09-13T05:02:40.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994900+0000", 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4698999999999999e-05 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994907+0000", 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2570000000000001e-06 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994952+0000", 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4885000000000001e-05 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.994971+0000", 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9093000000000002e-05 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996346+0000", 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001375287 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996379+0000", 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2700999999999997e-05 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.395 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10143 3.0 e29/19 3:0297533c:::rbd_data.117c6c62fa22.000000000000003f:head v 29'99, mlcod=28'93)", 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.376790909, 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001524173, 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995370+0000", 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995375+0000", 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8340000000000004e-06 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995387+0000", 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1046000000000001e-05 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995393+0000", 2024-09-13T05:02:40.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3509999999999998e-06 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995441+0000", 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7786e-05 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.995460+0000", 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9154e-05 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996857+0000", 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013971719999999999 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.996894+0000", 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7357999999999997e-05 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:19907 3.9 e31/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 31'115, mlcod=31'112)", 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.925317038999999, 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001592883, 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446843+0000", 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446850+0000", 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3450000000000002e-06 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446860+0000", 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446867+0000", 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3570000000000003e-06 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446923+0000", 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6242000000000001e-05 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446940+0000", 2024-09-13T05:02:40.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7209000000000001e-05 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448406+0000", 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001465383 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448437+0000", 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1372999999999999e-05 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:20095 3.9 e32/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 32'140, mlcod=32'137)", 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.176333277000001, 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014673279999999999, 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195828+0000", 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195827+0000", 2024-09-13T05:02:40.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195846+0000", 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8810000000000001e-05 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195860+0000", 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3835000000000001e-05 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195868+0000", 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2300000000000008e-06 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195910+0000", 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1690000000000002e-05 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.195938+0000", 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8829000000000001e-05 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.197231+0000", 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001292201 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.197295+0000", 2024-09-13T05:02:40.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4498000000000005e-05 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:20115 3.0 e32/19 3:07e79267:::rbd_data.117c3521aa89.000000000000000a:head v 32'193, mlcod=31'190)", 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.169542543999999, 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015367919999999999, 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.411 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202619+0000", 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202618+0000", 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202626+0000", 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0700000000000007e-06 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202636+0000", 2024-09-13T05:02:40.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0227000000000001e-05 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202645+0000", 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8040000000000007e-06 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202687+0000", 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2360000000000001e-05 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.202706+0000", 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8933e-05 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.204117+0000", 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014105089999999999 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:23.204155+0000", 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8544999999999998e-05 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 12.568713143, 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:02:40.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:02:40.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:40.421 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.4276441089999992, 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:02:40.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:02:40.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.1961234590000007, 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:02:40.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:02:40.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.1951477859999997, 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:02:40.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:02:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:02:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.1948269089999997, 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:40.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:02:40.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.1927389519999991, 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40675 3.4 e37/19 3:25c10db2:::rbd_data.117c4a06f7bc.000000000000024b:head v 37'226, mlcod=37'224)", 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.1890323400000007, 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001577501, 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:40.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183129+0000", 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183128+0000", 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183134+0000", 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.387e-06 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183149+0000", 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4664000000000001e-05 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183156+0000", 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9700000000000002e-06 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183436+0000", 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00027994199999999998 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.183455+0000", 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9287999999999999e-05 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184676+0000", 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001220308 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.184706+0000", 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639000000000003e-05 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40683 3.4 e37/19 3:2071a6c5:::rbd_data.117c4a06f7bc.000000000000025d:head v 37'227, mlcod=37'225)", 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.186869197, 2024-09-13T05:02:40.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001476371, 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:40.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185292+0000", 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185291+0000", 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185298+0000", 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185313+0000", 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5406000000000001e-05 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185326+0000", 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2313999999999999e-05 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185431+0000", 2024-09-13T05:02:40.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010521799999999999 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.185450+0000", 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9315000000000001e-05 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.186710+0000", 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012597730000000001 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.186768+0000", 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8236000000000003e-05 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41018 3.0 e38/19 3:004a91c0:::rbd_data.117c4a06f7bc.0000000000000274:head v 38'260, mlcod=37'256)", 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:40.462 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3156228409999997, 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001578716, 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056538+0000", 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.464 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056544+0000", 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5969999999999997e-06 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056558+0000", 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3900000000000001e-05 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056564+0000", 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4300000000000003e-06 2024-09-13T05:02:40.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056627+0000", 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2762999999999995e-05 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.056649+0000", 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2086000000000002e-05 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.058077+0000", 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014276460000000001 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.058117+0000", 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0014e-05 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.467 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.0984297769999998, 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:02:40.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:02:40.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:02:40.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:02:40.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.473 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.7932913159999999, 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:02:40.474 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:02:40.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:50973 3.4 e41/19 3:261ff264:::rbd_object_map.117cbeb44a94:head v 41'309, mlcod=41'307)", 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.737797+0000", 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 2.6343643380000001, 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014716340000000001, 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737797+0000", 2024-09-13T05:02:40.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737797+0000", 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737795+0000", 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737807+0000", 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1207e-05 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737827+0000", 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0055999999999999e-05 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737835+0000", 2024-09-13T05:02:40.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0090000000000007e-06 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737883+0000", 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8708000000000002e-05 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.737911+0000", 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7596000000000001e-05 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.739236+0000", 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013245399999999999 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.739268+0000", 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2808000000000002e-05 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.484 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.485 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.485 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:40.486 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:40.487 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:40.487 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:40.488 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:40.484+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:40.496 INFO:teuthology.orchestra.run.smithi053.stdout:3197 read 0x92c5569 thru 0x92cf029 (0x9ac1 bytes) 2024-09-13T05:02:40.496 INFO:teuthology.orchestra.run.smithi053.stdout:3198 punch from 0x9bbfa82 to 0x9bc363e, (0x3bbc bytes) 2024-09-13T05:02:40.502 INFO:teuthology.orchestra.run.smithi053.stdout:3199 write 0xd2a2edd thru 0xd2a4a00 (0x1b24 bytes) 2024-09-13T05:02:40.505 INFO:teuthology.orchestra.run.smithi053.stdout:3200 write 0x7fcbe9d thru 0x7fd2a9f (0x6c03 bytes) 2024-09-13T05:02:40.510 INFO:teuthology.orchestra.run.smithi053.stdout:3201 write 0xce985c7 thru 0xcea5ad3 (0xd50d bytes) 2024-09-13T05:02:40.514 INFO:teuthology.orchestra.run.smithi053.stdout:3202 read 0x9898ad thru 0x9913e9 (0x7b3d bytes) 2024-09-13T05:02:40.514 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.514 INFO:teuthology.orchestra.run.smithi053.stdout:3205 punch from 0x7b5968b to 0x7b663de, (0xcd53 bytes) 2024-09-13T05:02:40.514 INFO:teuthology.orchestra.run.smithi053.stdout:3206 writesame 0x699776b thru 0x699b63d data_size 0x3(0x3ed3 bytes) 2024-09-13T05:02:40.519 INFO:teuthology.orchestra.run.smithi053.stdout:3207 read 0xa79cfa2 thru 0xa7a4379 (0x73d8 bytes) 2024-09-13T05:02:40.519 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.519 INFO:teuthology.orchestra.run.smithi053.stdout:3210 writesame 0x83d7aea thru 0x83dcc35 data_size 0x2c(0x514c bytes) 2024-09-13T05:02:40.522 INFO:teuthology.orchestra.run.smithi053.stdout:3212 write 0xad5272c thru 0xad52aba (0x38f bytes) 2024-09-13T05:02:40.526 INFO:teuthology.orchestra.run.smithi053.stdout:3213 write 0x5b3ec00 thru 0x5b4ea09 (0xfe0a bytes) 2024-09-13T05:02:40.529 INFO:teuthology.orchestra.run.smithi053.stdout:3214 write 0xc6774c5 thru 0xc678813 (0x134f bytes) 2024-09-13T05:02:40.532 INFO:teuthology.orchestra.run.smithi053.stdout:3215 write 0xbd10b8e thru 0xbd144a7 (0x391a bytes) 2024-09-13T05:02:40.535 INFO:teuthology.orchestra.run.smithi053.stdout:3216 writesame 0xcdeabd9 thru 0xcdf9058 data_size 0x40(0xe480 bytes) 2024-09-13T05:02:40.540 INFO:teuthology.orchestra.run.smithi053.stdout:3217 punch from 0x39947c8 to 0x399779a, (0x2fd2 bytes) 2024-09-13T05:02:40.540 INFO:teuthology.orchestra.run.smithi053.stdout:3218 read 0x8160e52 thru 0x816fdec (0xef9b bytes) 2024-09-13T05:02:40.540 INFO:teuthology.orchestra.run.smithi053.stdout:3224 writesame 0x1fccc4a thru 0x1fce4c7 data_size 0x26(0x187e bytes) 2024-09-13T05:02:40.543 INFO:teuthology.orchestra.run.smithi053.stdout:3225 trunc from 0xd2a4a01 to 0xdd544b9 2024-09-13T05:02:40.549 INFO:teuthology.orchestra.run.smithi053.stdout:3226 punch from 0x2f905b6 to 0x2f9cde7, (0xc831 bytes) 2024-09-13T05:02:40.549 INFO:teuthology.orchestra.run.smithi053.stdout:3227 read 0x878af72 thru 0x879ad3c (0xfdcb bytes) 2024-09-13T05:02:40.554 INFO:teuthology.orchestra.run.smithi053.stdout:3228 write 0xdf919f7 thru 0xdf9d840 (0xbe4a bytes) 2024-09-13T05:02:40.558 INFO:teuthology.orchestra.run.smithi053.stdout:3229 read 0x927d539 thru 0x9283eb9 (0x6981 bytes) 2024-09-13T05:02:40.558 INFO:teuthology.orchestra.run.smithi053.stdout:3231 punch from 0xb675064 to 0xb681ec0, (0xce5c bytes) 2024-09-13T05:02:40.558 INFO:teuthology.orchestra.run.smithi053.stdout:3232 writesame 0x52530b thru 0x528945 data_size 0x1(0x363b bytes) 2024-09-13T05:02:40.572 INFO:teuthology.orchestra.run.smithi053.stdout:3233 trunc from 0xdf9d841 to 0x8b24225 2024-09-13T05:02:40.583 INFO:teuthology.orchestra.run.smithi053.stdout:3234 read 0x5df718 thru 0x5e247c (0x2d65 bytes) 2024-09-13T05:02:40.584 INFO:teuthology.orchestra.run.smithi053.stdout:3235 punch from 0x1ede4c to 0x1eeaee, (0xca2 bytes) 2024-09-13T05:02:40.584 INFO:teuthology.orchestra.run.smithi053.stdout:3236 write 0x433ba48 thru 0x434158e (0x5b47 bytes) 2024-09-13T05:02:40.587 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:40.586+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:40.588 INFO:teuthology.orchestra.run.smithi053.stdout:3237 punch from 0x2494d53 to 0x249ccfb, (0x7fa8 bytes) 2024-09-13T05:02:40.588 INFO:teuthology.orchestra.run.smithi053.stdout:3238 write 0x6f8e4c0 thru 0x6f8eae1 (0x622 bytes) 2024-09-13T05:02:40.601 INFO:teuthology.orchestra.run.smithi053.stdout:3240 writesame 0xc715a6d thru 0xc721ff4 data_size 0x62(0xc588 bytes) 2024-09-13T05:02:40.601 INFO:teuthology.orchestra.run.smithi053.stdout:3241 read 0xba31218 thru 0xba40701 (0xf4ea bytes) 2024-09-13T05:02:40.601 INFO:teuthology.orchestra.run.smithi053.stdout:3244 writesame 0xc404950 thru 0xc40db6d data_size 0x3b(0x921e bytes) 2024-09-13T05:02:40.604 INFO:teuthology.orchestra.run.smithi053.stdout:3245 writesame 0x83832ef thru 0x8389ca0 data_size 0x2(0x69b2 bytes) 2024-09-13T05:02:40.612 INFO:teuthology.orchestra.run.smithi053.stdout:3247 read 0x917f7d5 thru 0x918d0b5 (0xd8e1 bytes) 2024-09-13T05:02:40.612 INFO:teuthology.orchestra.run.smithi053.stdout:3248 punch from 0x571e7a8 to 0x572c776, (0xdfce bytes) 2024-09-13T05:02:40.613 INFO:teuthology.orchestra.run.smithi053.stdout:3249 write 0x479c5dc thru 0x47aa727 (0xe14c bytes) 2024-09-13T05:02:40.617 INFO:teuthology.orchestra.run.smithi053.stdout:3252 read 0xa664d5 thru 0xa6dab5 (0x75e1 bytes) 2024-09-13T05:02:40.617 INFO:teuthology.orchestra.run.smithi053.stdout:3253 writesame 0xa1f81b8 thru 0xa1f926d data_size 0x2e(0x10b6 bytes) 2024-09-13T05:02:40.620 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:40.620 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:40.620 INFO:teuthology.orchestra.run.smithi053.stdout:3254 write 0x58bb82e thru 0x58bddfb (0x25ce bytes) 2024-09-13T05:02:40.623 INFO:teuthology.orchestra.run.smithi053.stdout:3256 punch from 0x6ba4e92 to 0x6babe3e, (0x6fac bytes) 2024-09-13T05:02:40.623 INFO:teuthology.orchestra.run.smithi053.stdout:3257 writesame 0xb3cbe7e thru 0xb3dade1 data_size 0x6(0xef64 bytes) 2024-09-13T05:02:40.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:40.630 INFO:teuthology.orchestra.run.smithi053.stdout:3258 write 0x477efc4 thru 0x478de71 (0xeeae bytes) 2024-09-13T05:02:40.635 INFO:teuthology.orchestra.run.smithi053.stdout:3260 trunc from 0xc721ff5 to 0xd65499f 2024-09-13T05:02:40.647 INFO:teuthology.orchestra.run.smithi053.stdout:3262 write 0xd858e61 thru 0xd85e206 (0x53a6 bytes) 2024-09-13T05:02:40.650 INFO:teuthology.orchestra.run.smithi053.stdout:3263 write 0x45a37c thru 0x465c30 (0xb8b5 bytes) 2024-09-13T05:02:40.653 INFO:teuthology.orchestra.run.smithi053.stdout:3264 write 0x5b2acbb thru 0x5b39ffd (0xf343 bytes) 2024-09-13T05:02:40.656 INFO:teuthology.orchestra.run.smithi053.stdout:3265 write 0x86d652d thru 0x86d71c0 (0xc94 bytes) 2024-09-13T05:02:40.668 INFO:teuthology.orchestra.run.smithi053.stdout:3266 read 0xaac266e thru 0xaac2bc4 (0x557 bytes) 2024-09-13T05:02:40.672 INFO:teuthology.orchestra.run.smithi053.stdout:3267 punch from 0x19760b8 to 0x1985d54, (0xfc9c bytes) 2024-09-13T05:02:40.672 INFO:teuthology.orchestra.run.smithi053.stdout:3268 read 0x4b0fba2 thru 0x4b1f5c4 (0xfa23 bytes) 2024-09-13T05:02:40.672 INFO:teuthology.orchestra.run.smithi053.stdout:3269 writesame 0xba248 thru 0xc41b4 data_size 0x1(0x9f6d bytes) 2024-09-13T05:02:40.688 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:40.687+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:40.694 INFO:teuthology.orchestra.run.smithi053.stdout:3270 trunc from 0xd85e207 to 0x35ebfbe 2024-09-13T05:02:40.710 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.710 INFO:teuthology.orchestra.run.smithi053.stdout:3272 punch from 0x22eec42 to 0x22f8f0c, (0xa2ca bytes) 2024-09-13T05:02:40.720 INFO:teuthology.orchestra.run.smithi053.stdout:3275 trunc from 0x35ebfbe to 0xbbe9613 2024-09-13T05:02:40.726 INFO:teuthology.orchestra.run.smithi053.stdout:3276 punch from 0x22f4d3 to 0x23964f, (0xa17c bytes) 2024-09-13T05:02:40.726 INFO:teuthology.orchestra.run.smithi053.stdout:3277 read 0x7062db3 thru 0x706d260 (0xa4ae bytes) 2024-09-13T05:02:40.727 INFO:teuthology.orchestra.run.smithi053.stdout:3278 write 0x1c382ff thru 0x1c3e711 (0x6413 bytes) 2024-09-13T05:02:40.729 INFO:teuthology.orchestra.run.smithi053.stdout:3279 write 0x8de6929 thru 0x8df372f (0xce07 bytes) 2024-09-13T05:02:40.744 INFO:teuthology.orchestra.run.smithi053.stdout:3280 trunc from 0xbbe9613 to 0x295dc08 2024-09-13T05:02:40.759 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:40.759 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:40.759 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:40.760 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:40.760 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:40.772 INFO:teuthology.orchestra.run.smithi053.stdout:3281 write 0xe1b0094 thru 0xe1b9234 (0x91a1 bytes) 2024-09-13T05:02:40.774 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:40.791 INFO:teuthology.orchestra.run.smithi053.stdout:3282 trunc from 0xe1b9235 to 0xac2022 2024-09-13T05:02:40.791 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:40.789+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:40.817 INFO:teuthology.orchestra.run.smithi053.stdout:3283 write 0x5b8e895 thru 0x5b8fc21 (0x138d bytes) 2024-09-13T05:02:40.820 INFO:teuthology.orchestra.run.smithi053.stdout:3284 read 0x327f3fd thru 0x32865f2 (0x71f6 bytes) 2024-09-13T05:02:40.820 INFO:teuthology.orchestra.run.smithi053.stdout:3285 write 0x58f1282 thru 0x58f969c (0x841b bytes) 2024-09-13T05:02:40.823 INFO:teuthology.orchestra.run.smithi053.stdout:3286 writesame 0x2ef74f6 thru 0x2ef8337 data_size 0xa(0xe42 bytes) 2024-09-13T05:02:40.826 INFO:teuthology.orchestra.run.smithi053.stdout:3287 read 0x451f885 thru 0x4520057 (0x7d3 bytes) 2024-09-13T05:02:40.826 INFO:teuthology.orchestra.run.smithi053.stdout:3288 read 0x3eaba47 thru 0x3eb6499 (0xaa53 bytes) 2024-09-13T05:02:40.826 INFO:teuthology.orchestra.run.smithi053.stdout:3289 punch from 0x35e0255 to 0x35e2c36, (0x29e1 bytes) 2024-09-13T05:02:40.841 INFO:teuthology.orchestra.run.smithi053.stdout:3290 write 0xe01df0f thru 0xe027a75 (0x9b67 bytes) 2024-09-13T05:02:40.845 INFO:teuthology.orchestra.run.smithi053.stdout:3291 read 0x1880287 thru 0x1880edd (0xc57 bytes) 2024-09-13T05:02:40.845 INFO:teuthology.orchestra.run.smithi053.stdout:3294 punch from 0x66c8d70 to 0x66d51c6, (0xc456 bytes) 2024-09-13T05:02:40.845 INFO:teuthology.orchestra.run.smithi053.stdout:3295 write 0xb56eecc thru 0xb57603d (0x7172 bytes) 2024-09-13T05:02:40.850 INFO:teuthology.orchestra.run.smithi053.stdout:3296 write 0x9ed0a72 thru 0x9edd37e (0xc90d bytes) 2024-09-13T05:02:40.852 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.852 INFO:teuthology.orchestra.run.smithi053.stdout:3298 write 0x3a0e64f thru 0x3a17799 (0x914b bytes) 2024-09-13T05:02:40.856 INFO:teuthology.orchestra.run.smithi053.stdout:3299 write 0x5db8f44 thru 0x5dbb803 (0x28c0 bytes) 2024-09-13T05:02:40.874 INFO:teuthology.orchestra.run.smithi053.stdout:3302 trunc from 0xe027a76 to 0xfcb37c 2024-09-13T05:02:40.886 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.886 INFO:teuthology.orchestra.run.smithi053.stdout:3306 read 0x592012 thru 0x59362b (0x161a bytes) 2024-09-13T05:02:40.886 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.890 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:40.889+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:40.907 INFO:teuthology.orchestra.run.smithi053.stdout:3309 write 0xe64b62c thru 0xe65701f (0xb9f4 bytes) 2024-09-13T05:02:40.911 INFO:teuthology.orchestra.run.smithi053.stdout:3310 write 0x2318d3c thru 0x232076a (0x7a2f bytes) 2024-09-13T05:02:40.915 INFO:teuthology.orchestra.run.smithi053.stdout:3311 read 0xabf7333 thru 0xabfe95d (0x762b bytes) 2024-09-13T05:02:40.915 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.915 INFO:teuthology.orchestra.run.smithi053.stdout:3313 read 0x55d4c6d thru 0x55d6bca (0x1f5e bytes) 2024-09-13T05:02:40.915 INFO:teuthology.orchestra.run.smithi053.stdout:3316 write 0xbeff3f3 thru 0xbf039cc (0x45da bytes) 2024-09-13T05:02:40.918 INFO:teuthology.orchestra.run.smithi053.stdout:3317 punch from 0x71e6ce5 to 0x71f2d7e, (0xc099 bytes) 2024-09-13T05:02:40.918 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:40.919 INFO:teuthology.orchestra.run.smithi053.stdout:3320 read 0xc15b0bc thru 0xc1694b7 (0xe3fc bytes) 2024-09-13T05:02:40.919 INFO:teuthology.orchestra.run.smithi053.stdout:3321 write 0x77d826b thru 0x77e06dd (0x8473 bytes) 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:40.922 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.402334891999999, 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:40.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:02:40.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:02:40.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.402163587, 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:02:40.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:40.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:02:40.934 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47845 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1564672~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.365919+0000", 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5543103380000001, 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017669296000000001, 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.935 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47845 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365919+0000", 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365919+0000", 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365918+0000", 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365921+0000", 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365929+0000", 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.678e-06 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365935+0000", 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383394+0000", 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017459005999999999 2024-09-13T05:02:40.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383485+0000", 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1011e-05 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383588+0000", 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010336900000000001 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48085 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 598016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368354+0000", 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.551875173, 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017542979, 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48085 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.940 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368354+0000", 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368354+0000", 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368353+0000", 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368358+0000", 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0200000000000002e-06 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368367+0000", 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7369999999999995e-06 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368371+0000", 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2390000000000004e-06 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385707+0000", 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017335910999999999 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385805+0000", 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8047999999999994e-05 2024-09-13T05:02:40.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385897+0000", 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1710000000000001e-05 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48086 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5518461540000001, 2024-09-13T05:02:40.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017753149999999999, 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48086 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368384+0000", 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.829e-06 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368388+0000", 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000002e-06 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368392+0000", 2024-09-13T05:02:40.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5300000000000001e-06 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385946+0000", 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017554422 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386034+0000", 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7174000000000003e-05 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386136+0000", 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010249799999999999 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48087 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368569+0000", 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5516600240000002, 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017776863, 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48087 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368569+0000", 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368569+0000", 2024-09-13T05:02:40.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368568+0000", 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368573+0000", 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3259999999999997e-06 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368584+0000", 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1394e-05 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368589+0000", 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9219999999999996e-06 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386166+0000", 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017577354 2024-09-13T05:02:40.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386250+0000", 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4167999999999998e-05 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386346+0000", 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5478999999999998e-05 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48088 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368601+0000", 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5516284250000001, 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017953779999999999, 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48088 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368601+0000", 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368601+0000", 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368600+0000", 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368602+0000", 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2000000000000001e-06 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368607+0000", 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2100000000000003e-06 2024-09-13T05:02:40.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368609+0000", 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.841e-06 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386388+0000", 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017778828999999999 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386451+0000", 2024-09-13T05:02:40.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3146999999999997e-05 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386554+0000", 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102942 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.958 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48089 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368690+0000", 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5515390760000001, 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018046405000000001, 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48089 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368690+0000", 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368690+0000", 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368689+0000", 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:40.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368694+0000", 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0810000000000004e-06 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368702+0000", 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4339999999999992e-06 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368708+0000", 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7359999999999998e-06 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386596+0000", 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017887888000000001 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386680+0000", 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4277000000000006e-05 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386736+0000", 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6523000000000001e-05 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48090 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368820+0000", 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.551408742, 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018075834999999998, 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.963 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48090 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368820+0000", 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368820+0000", 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368819+0000", 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368824+0000", 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8060000000000002e-06 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368835+0000", 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0403e-05 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368843+0000", 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1210000000000007e-06 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386766+0000", 2024-09-13T05:02:40.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017923005999999998 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386820+0000", 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4191999999999999e-05 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386896+0000", 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6147000000000001e-05 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48091 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 950272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368896+0000", 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5513328639999999, 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018221838000000001, 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48091 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368896+0000", 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368896+0000", 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368895+0000", 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368899+0000", 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0119999999999999e-06 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368908+0000", 2024-09-13T05:02:40.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8319999999999995e-06 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368914+0000", 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9290000000000003e-06 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386937+0000", 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01802287 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387026+0000", 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9347999999999999e-05 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387118+0000", 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1694000000000007e-05 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48092 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.972 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5511686230000001, 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018282850999999999, 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48092 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369064+0000", 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6589999999999998e-06 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369074+0000", 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369080+0000", 2024-09-13T05:02:40.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6320000000000002e-06 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387156+0000", 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018075872999999999 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387246+0000", 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9963999999999994e-05 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387343+0000", 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7156000000000003e-05 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48093 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1048576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5509686440000001, 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018225456000000001, 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.977 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48093 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369264+0000", 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369274+0000", 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4970000000000007e-06 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369280+0000", 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4400000000000002e-06 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387381+0000", 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018101092999999999 2024-09-13T05:02:40.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387433+0000", 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1823000000000002e-05 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387486+0000", 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2704000000000002e-05 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48094 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5507986799999998, 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018210301000000002, 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48094 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.982 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369434+0000", 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5349999999999998e-06 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369446+0000", 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1445e-05 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369466+0000", 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0044999999999999e-05 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387524+0000", 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018058576 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387580+0000", 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5550999999999997e-05 2024-09-13T05:02:40.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387641+0000", 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0965999999999999e-05 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48095 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1163264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.550567542, 2024-09-13T05:02:40.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018110261999999999, 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48095 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369666+0000", 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3669999999999996e-06 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369677+0000", 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0873000000000001e-05 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369685+0000", 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1689999999999992e-06 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387679+0000", 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017993776999999999 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387720+0000", 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0929000000000001e-05 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387772+0000", 2024-09-13T05:02:40.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2108000000000003e-05 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48096 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369909+0000", 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5503199620000001, 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018114216999999998, 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48096 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369909+0000", 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369909+0000", 2024-09-13T05:02:40.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369908+0000", 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369913+0000", 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4970000000000003e-06 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369922+0000", 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2399999999999996e-06 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369928+0000", 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4979999999999997e-06 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387845+0000", 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017917807000000001 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387930+0000", 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4864999999999995e-05 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388023+0000", 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3066000000000006e-05 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.995 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48097 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1261568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370061+0000", 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5501678500000002, 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018148152000000001, 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48097 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:40.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370061+0000", 2024-09-13T05:02:40.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370061+0000", 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370060+0000", 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370066+0000", 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9869999999999996e-06 2024-09-13T05:02:40.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370077+0000", 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0844e-05 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370084+0000", 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5150000000000003e-06 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:40.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388066+0000", 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017981967000000001 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388149+0000", 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3150000000000002e-05 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388209+0000", 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0448999999999997e-05 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48098 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370102+0000", 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5501274499999997, 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018247026999999999, 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48098 2024-09-13T05:02:41.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.002 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:41.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:41.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370102+0000", 2024-09-13T05:02:41.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370102+0000", 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370101+0000", 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:41.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370105+0000", 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8469999999999998e-06 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370112+0000", 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370118+0000", 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0610000000000001e-06 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388238+0000", 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018119988 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388291+0000", 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2264e-05 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388349+0000", 2024-09-13T05:02:41.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7865999999999998e-05 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48099 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370124+0000", 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5501051320000001, 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018485003999999999, 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:41.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48099 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370124+0000", 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:41.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370124+0000", 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370123+0000", 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370126+0000", 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.61e-06 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370133+0000", 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6869999999999997e-06 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370138+0000", 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0420000000000002e-06 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388400+0000", 2024-09-13T05:02:41.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018262332999999999 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388501+0000", 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010084 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388609+0000", 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010792400000000001 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48100 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5500855929999999, 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018661594, 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48100 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:41.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:41.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370145+0000", 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370151+0000", 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0220000000000001e-06 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370156+0000", 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9729999999999997e-06 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388650+0000", 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018493451000000001 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388704+0000", 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4401000000000003e-05 2024-09-13T05:02:41.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388805+0000", 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010076 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48101 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5500650760000001, 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018784986, 2024-09-13T05:02:41.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48101 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370166+0000", 2024-09-13T05:02:41.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370173+0000", 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-06 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370177+0000", 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9869999999999999e-06 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:41.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388844+0000", 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018666312000000001 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388893+0000", 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9604000000000001e-05 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388949+0000", 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5529000000000003e-05 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:41.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:41.020 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:41.020 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:41.020 INFO:teuthology.orchestra.run.smithi053.stdout:3324 write 0x96746ed thru 0x9679259 (0x4b6d bytes) 2024-09-13T05:02:41.021 INFO:teuthology.orchestra.run.smithi053.stdout:3325 write 0x958f178 thru 0x9590183 (0x100c bytes) 2024-09-13T05:02:41.021 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:41.021 INFO:teuthology.orchestra.run.smithi053.stdout:3327 write 0x9ecc1f6 thru 0x9ecebf3 (0x29fe bytes) 2024-09-13T05:02:41.022 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:41.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3329 read 0x1d6674c thru 0x1d6c57c (0x5e31 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3330 punch from 0x985ef7c to 0x9865781, (0x6805 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3331 write 0x92e7185 thru 0x92e85d4 (0x1450 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3333 trunc from 0xe657020 to 0xb98a0cf 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3334 punch from 0x64e064a to 0x64f060d, (0xffc3 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3335 punch from 0x54395a6 to 0x5444bbd, (0xb617 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3342 punch from 0x6e527f7 to 0x6e583bc, (0x5bc5 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3344 trunc from 0xb98a0cf to 0xb47a79d 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3346 write 0xd947c39 thru 0xd9542b0 (0xc678 bytes) 2024-09-13T05:02:41.025 INFO:teuthology.orchestra.run.smithi053.stdout:3347 punch from 0xbce1a43 to 0xbcf0384, (0xe941 bytes) 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3348 write 0xe8eb290 thru 0xe8ebca3 (0xa14 bytes) 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3351 punch from 0x5dfce42 to 0x5e0c600, (0xf7be bytes) 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3352 write 0xbc693ea thru 0xbc78844 (0xf45b bytes) 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3354 trunc from 0xe8ebca4 to 0x71c7a54 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3355 trunc from 0x71c7a54 to 0xd21110f 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3357 write 0xaeca8fe thru 0xaed1708 (0x6e0b bytes) 2024-09-13T05:02:41.026 INFO:teuthology.orchestra.run.smithi053.stdout:3359 write 0xda66a21 thru 0xda6ca53 (0x6033 bytes) 2024-09-13T05:02:41.038 INFO:teuthology.orchestra.run.smithi053.stdout:3360 trunc from 0xda6ca54 to 0x243cea 2024-09-13T05:02:41.039 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:41.036+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:41.061 INFO:teuthology.orchestra.run.smithi053.stdout:3362 write 0x35ca8a7 thru 0x35cc440 (0x1b9a bytes) 2024-09-13T05:02:41.064 INFO:teuthology.orchestra.run.smithi053.stdout:3363 punch from 0x24eaa76 to 0x24f8642, (0xdbcc bytes) 2024-09-13T05:02:41.064 INFO:teuthology.orchestra.run.smithi053.stdout:3364 read 0x2c8a8cd thru 0x2c90a0d (0x6141 bytes) 2024-09-13T05:02:41.070 INFO:teuthology.orchestra.run.smithi053.stdout:3365 write 0x490d5bc thru 0x491369a (0x60df bytes) 2024-09-13T05:02:41.074 INFO:teuthology.orchestra.run.smithi053.stdout:3366 read 0x4648ab3 thru 0x4650b29 (0x8077 bytes) 2024-09-13T05:02:41.074 INFO:teuthology.orchestra.run.smithi053.stdout:3367 write 0x6ed554 thru 0x6f875e (0xb20b bytes) 2024-09-13T05:02:41.079 INFO:teuthology.orchestra.run.smithi053.stdout:3368 read 0x11e2e89 thru 0x11e8e67 (0x5fdf bytes) 2024-09-13T05:02:41.079 INFO:teuthology.orchestra.run.smithi053.stdout:3369 read 0x2b81600 thru 0x2b8eb14 (0xd515 bytes) 2024-09-13T05:02:41.084 INFO:teuthology.orchestra.run.smithi053.stdout:3370 write 0x4d4e087 thru 0x4d5a82a (0xc7a4 bytes) 2024-09-13T05:02:41.089 INFO:teuthology.orchestra.run.smithi053.stdout:3371 read 0xfc8b0b thru 0xfd5a74 (0xcf6a bytes) 2024-09-13T05:02:41.089 INFO:teuthology.orchestra.run.smithi053.stdout:3372 punch from 0x3bcb74f to 0x3bd2c02, (0x74b3 bytes) 2024-09-13T05:02:41.089 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:41.089 INFO:teuthology.orchestra.run.smithi053.stdout:3375 read 0x1580d83 thru 0x1590b00 (0xfd7e bytes) 2024-09-13T05:02:41.098 INFO:teuthology.orchestra.run.smithi053.stdout:3377 write 0x7cc05b0 thru 0x7cca9c6 (0xa417 bytes) 2024-09-13T05:02:41.101 INFO:teuthology.orchestra.run.smithi053.stdout:3379 read 0x3526602 thru 0x3533ad8 (0xd4d7 bytes) 2024-09-13T05:02:41.101 INFO:teuthology.orchestra.run.smithi053.stdout:3381 punch from 0x57a2419 to 0x57a2748, (0x32f bytes) 2024-09-13T05:02:41.109 INFO:teuthology.orchestra.run.smithi053.stdout:3384 writesame 0x98703ac thru 0x987514c data_size 0x11(0x4da1 bytes) 2024-09-13T05:02:41.109 INFO:teuthology.orchestra.run.smithi053.stdout:3385 read 0x73e1634 thru 0x73ee58a (0xcf57 bytes) 2024-09-13T05:02:41.109 INFO:teuthology.orchestra.run.smithi053.stdout:3386 read 0x69a411f thru 0x69a7975 (0x3857 bytes) 2024-09-13T05:02:41.109 INFO:teuthology.orchestra.run.smithi053.stdout:3387 write 0x120ddfc thru 0x1216697 (0x889c bytes) 2024-09-13T05:02:41.113 INFO:teuthology.orchestra.run.smithi053.stdout:3388 punch from 0x7fc746a to 0x7fcf1ad, (0x7d43 bytes) 2024-09-13T05:02:41.120 INFO:teuthology.orchestra.run.smithi053.stdout:3389 write 0xb1fbbec thru 0xb1fc98d (0xda2 bytes) 2024-09-13T05:02:41.123 INFO:teuthology.orchestra.run.smithi053.stdout:3390 punch from 0x4cfa35 to 0x4db0ee, (0xb6b9 bytes) 2024-09-13T05:02:41.123 INFO:teuthology.orchestra.run.smithi053.stdout:3391 write 0x3461d74 thru 0x3465f0d (0x419a bytes) 2024-09-13T05:02:41.124 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:41.123+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:41.126 INFO:teuthology.orchestra.run.smithi053.stdout:3393 write 0x39aa5e4 thru 0x39b1411 (0x6e2e bytes) 2024-09-13T05:02:41.130 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:41.131 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:41.131 INFO:teuthology.orchestra.run.smithi053.stdout:3394 read 0x4308e6d thru 0x4318299 (0xf42d bytes) 2024-09-13T05:02:41.131 INFO:teuthology.orchestra.run.smithi053.stdout:3399 punch from 0x27197bf to 0x271cd39, (0x357a bytes) 2024-09-13T05:02:41.139 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:41.140 INFO:teuthology.orchestra.run.smithi053.stdout:3401 write 0xe64daac thru 0xe64f921 (0x1e76 bytes) 2024-09-13T05:02:41.143 INFO:teuthology.orchestra.run.smithi053.stdout:3402 writesame 0x371ef43 thru 0x3721bb6 data_size 0xa(0x2c74 bytes) 2024-09-13T05:02:41.147 INFO:teuthology.orchestra.run.smithi053.stdout:3403 writesame 0xb9cf896 thru 0xb9d49e3 data_size 0x3(0x514e bytes) 2024-09-13T05:02:41.153 INFO:teuthology.orchestra.run.smithi053.stdout:3404 writesame 0x2adb629 thru 0x2ae787e data_size 0x32(0xc256 bytes) 2024-09-13T05:02:41.157 INFO:teuthology.orchestra.run.smithi053.stdout:3405 write 0xbe5d428 thru 0xbe6d1ad (0xfd86 bytes) 2024-09-13T05:02:41.161 INFO:teuthology.orchestra.run.smithi053.stdout:3407 punch from 0xa28e7fe to 0xa29cb30, (0xe332 bytes) 2024-09-13T05:02:41.161 INFO:teuthology.orchestra.run.smithi053.stdout:3408 read 0x7867817 thru 0x7874c46 (0xd430 bytes) 2024-09-13T05:02:41.161 INFO:teuthology.orchestra.run.smithi053.stdout:3409 trunc from 0xe64f922 to 0xe3c0aa2 2024-09-13T05:02:41.172 INFO:teuthology.orchestra.run.smithi053.stdout:3410 read 0xbc18b33 thru 0xbc1e4e6 (0x59b4 bytes) 2024-09-13T05:02:41.172 INFO:teuthology.orchestra.run.smithi053.stdout:3411 trunc from 0xe3c0aa2 to 0xec2c687 2024-09-13T05:02:41.177 INFO:teuthology.orchestra.run.smithi053.stdout:3412 write 0x4d6cc2d thru 0x4d7143b (0x480f bytes) 2024-09-13T05:02:41.181 INFO:teuthology.orchestra.run.smithi053.stdout:3413 read 0xb8dcc70 thru 0xb8e83af (0xb740 bytes) 2024-09-13T05:02:41.190 INFO:teuthology.orchestra.run.smithi053.stdout:3415 trunc from 0xec2c687 to 0x7104e42 2024-09-13T05:02:41.200 INFO:teuthology.orchestra.run.smithi053.stdout:3416 read 0x4ae31d thru 0x4b4cd5 (0x69b9 bytes) 2024-09-13T05:02:41.206 INFO:teuthology.orchestra.run.smithi053.stdout:3420 write 0x80750f9 thru 0x8075296 (0x19e bytes) 2024-09-13T05:02:41.210 INFO:teuthology.orchestra.run.smithi053.stdout:3421 trunc from 0x8075297 to 0x81f99bd 2024-09-13T05:02:41.215 INFO:teuthology.orchestra.run.smithi053.stdout:3422 read 0x6f0bec5 thru 0x6f18d5c (0xce98 bytes) 2024-09-13T05:02:41.215 INFO:teuthology.orchestra.run.smithi053.stdout:3423 write 0x1a8bb07 thru 0x1a9af37 (0xf431 bytes) 2024-09-13T05:02:41.219 INFO:teuthology.orchestra.run.smithi053.stdout:3424 punch from 0x7943ab0 to 0x794af95, (0x74e5 bytes) 2024-09-13T05:02:41.219 INFO:teuthology.orchestra.run.smithi053.stdout:3425 write 0x816cd38 thru 0x816de3d (0x1106 bytes) 2024-09-13T05:02:41.222 INFO:teuthology.orchestra.run.smithi053.stdout:3426 read 0x83b616 thru 0x841203 (0x5bee bytes) 2024-09-13T05:02:41.222 INFO:teuthology.orchestra.run.smithi053.stdout:3427 read 0x2ed9d99 thru 0x2ee8637 (0xe89f bytes) 2024-09-13T05:02:41.223 INFO:teuthology.orchestra.run.smithi053.stdout:3428 punch from 0x19fe854 to 0x1a0ccae, (0xe45a bytes) 2024-09-13T05:02:41.223 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:41.223 INFO:teuthology.orchestra.run.smithi053.stdout:3431 read 0x26e5edb thru 0x26e77dc (0x1902 bytes) 2024-09-13T05:02:41.223 INFO:teuthology.orchestra.run.smithi053.stdout:3432 trunc from 0x81f99bd to 0x7fb2c34 2024-09-13T05:02:41.225 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:41.223+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:41.232 INFO:teuthology.orchestra.run.smithi053.stdout:3433 writesame 0x19f54be thru 0x19ff113 data_size 0x1(0x9c56 bytes) 2024-09-13T05:02:41.246 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:41.246 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:41.246 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:41.247 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:41.247 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:41.252 INFO:teuthology.orchestra.run.smithi053.stdout:3434 punch from 0x63ba8b3 to 0x63c160f, (0x6d5c bytes) 2024-09-13T05:02:41.255 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:41.256 INFO:teuthology.orchestra.run.smithi053.stdout:3435 trunc from 0x7fb2c34 to 0xaee9614 2024-09-13T05:02:41.261 INFO:teuthology.orchestra.run.smithi053.stdout:3438 punch from 0x2d736c0 to 0x2d7a39c, (0x6cdc bytes) 2024-09-13T05:02:41.270 INFO:teuthology.orchestra.run.smithi053.stdout:3439 writesame 0xddda4d7 thru 0xddde10b data_size 0x1(0x3c35 bytes) 2024-09-13T05:02:41.270 INFO:teuthology.orchestra.run.smithi053.stdout:3441 writesame 0x66419fd thru 0x664f5e2 data_size 0x7(0xdbe6 bytes) 2024-09-13T05:02:41.276 INFO:teuthology.orchestra.run.smithi053.stdout:3442 write 0x73207f2 thru 0x7323b26 (0x3335 bytes) 2024-09-13T05:02:41.279 INFO:teuthology.orchestra.run.smithi053.stdout:3444 read 0xcbe26a6 thru 0xcbe56cd (0x3028 bytes) 2024-09-13T05:02:41.279 INFO:teuthology.orchestra.run.smithi053.stdout:3445 clone 21 order 19 su 131072 sc 2 2024-09-13T05:02:41.326 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:41.325+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:41.398 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:41.398 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:41.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:41.398 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154601048, 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:02:41.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:02:41.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154584793, 2024-09-13T05:02:41.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.407 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154567704, 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:02:41.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:02:41.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154550837, 2024-09-13T05:02:41.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:02:41.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154501539, 2024-09-13T05:02:41.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:02:41.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.421 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.15447827, 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:02:41.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:02:41.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154462536, 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:02:41.426 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:02:41.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.430 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154439556, 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:02:41.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:02:41.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154390490000001, 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:02:41.435 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:02:41.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.439 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154269175, 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:02:41.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154213961, 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:02:41.444 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:02:41.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:02:41.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154193672, 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:41.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:02:41.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:41.453 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154176490999999, 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:02:41.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.457 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154088980999999, 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:02:41.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:02:41.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154069733, 2024-09-13T05:02:41.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:02:41.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:02:41.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.466 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.154008618000001, 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:02:41.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.153991443000001, 2024-09-13T05:02:41.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:02:41.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:02:41.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.153975033, 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.476 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.480 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.153954048999999, 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.484 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.153936625, 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:02:41.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.489 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:41.491 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:41.491 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:41.489+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:41.591 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:41.591+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:41.594 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:41.594 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:41.603 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:41.692 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:41.691+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:41.704 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:41.704 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:41.704 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:41.704 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:41.704 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:41.713 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:41.793 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:41.793+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:41.816 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:41.816 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:41.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:41.816 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30534 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194558+0000", 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.620195469, 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035531742999999998, 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30534 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194558+0000", 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194558+0000", 2024-09-13T05:02:41.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194557+0000", 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194560+0000", 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2689999999999998e-06 2024-09-13T05:02:41.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194566+0000", 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3289999999999999e-06 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194576+0000", 2024-09-13T05:02:41.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5310000000000007e-06 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.229910+0000", 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035333950000000003 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.229995+0000", 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230090+0000", 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4876000000000004e-05 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30535 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194581+0000", 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.620172673000001, 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035632084000000001, 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.822 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30535 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194581+0000", 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194581+0000", 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194580+0000", 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194582+0000", 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.154e-06 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194589+0000", 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3589999999999996e-06 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194595+0000", 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6779999999999997e-06 2024-09-13T05:02:41.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230117+0000", 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035522977999999997 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230158+0000", 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0664999999999998e-05 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230213+0000", 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.465e-05 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30536 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:41.827 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.620154369, 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035829891000000003, 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30536 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194601+0000", 2024-09-13T05:02:41.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0389999999999999e-06 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194607+0000", 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194612+0000", 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1610000000000002e-06 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230248+0000", 2024-09-13T05:02:41.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035635714999999998 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230331+0000", 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3089000000000004e-05 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230429+0000", 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8107000000000001e-05 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.831 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30537 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2138112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.620137041, 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036017699, 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30537 2024-09-13T05:02:41.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194618+0000", 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9889999999999999e-06 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194624+0000", 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0109999999999997e-06 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194636+0000", 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1738e-05 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230464+0000", 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035828477999999997 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230543+0000", 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8838000000000006e-05 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230634+0000", 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1013000000000004e-05 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.836 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30540 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2236416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194649+0000", 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.620104733, 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036213493999999999, 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30540 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.837 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194649+0000", 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194649+0000", 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194648+0000", 2024-09-13T05:02:41.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194650+0000", 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3029999999999998e-06 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194663+0000", 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3012e-05 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194671+0000", 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230675+0000", 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036004622 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230761+0000", 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5778000000000004e-05 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230862+0000", 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100936 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30542 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2334720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194676+0000", 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.620077796, 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036384622999999998, 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.841 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30542 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194676+0000", 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.842 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194676+0000", 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194675+0000", 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194677+0000", 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2299999999999998e-06 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194684+0000", 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3600000000000001e-06 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194688+0000", 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6190000000000002e-06 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230881+0000", 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036192584 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230961+0000", 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0389000000000006e-05 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.231060+0000", 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8808000000000007e-05 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.845 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30559 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2924544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196700+0000", 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.618053586, 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035438492000000002, 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.846 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30559 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196700+0000", 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196700+0000", 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196699+0000", 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196703+0000", 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.709e-06 2024-09-13T05:02:41.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196714+0000", 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1080999999999999e-05 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196721+0000", 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8140000000000003e-06 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232062+0000", 2024-09-13T05:02:41.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035340871000000003 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232099+0000", 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.68e-05 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232138+0000", 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9743999999999998e-05 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30561 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3022848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.618027073, 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035553504999999999, 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30561 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.851 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:41.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196729+0000", 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196735+0000", 2024-09-13T05:02:41.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4579999999999998e-06 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196740+0000", 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9180000000000002e-06 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232165+0000", 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035424663000000002 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232219+0000", 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4317000000000002e-05 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232280+0000", 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0871e-05 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30562 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3121152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196745+0000", 2024-09-13T05:02:41.855 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.618008745000001, 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035759104999999999, 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30562 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196745+0000", 2024-09-13T05:02:41.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196745+0000", 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196744+0000", 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196746+0000", 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5659999999999999e-06 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196750+0000", 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.636e-06 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196753+0000", 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.021e-06 2024-09-13T05:02:41.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232325+0000", 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035571924999999997 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232409+0000", 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4376999999999995e-05 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232504+0000", 2024-09-13T05:02:41.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4843e-05 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30564 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3153920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617998423, 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035867128999999998, 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30564 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196756+0000", 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196759+0000", 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.932e-06 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196767+0000", 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1110000000000008e-06 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232537+0000", 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035769586999999999 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232582+0000", 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5358000000000002e-05 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232622+0000", 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0057e-05 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30565 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617983326000001, 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035982617000000001, 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30565 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196772+0000", 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0389999999999999e-06 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196778+0000", 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2210000000000002e-06 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196782+0000", 2024-09-13T05:02:41.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4070000000000003e-06 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232647+0000", 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035864723000000001 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232707+0000", 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9416999999999997e-05 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232753+0000", 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6156999999999999e-05 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30566 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3350528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196787+0000", 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617966770000001, 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036159697999999997, 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.869 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30566 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196787+0000", 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196787+0000", 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196786+0000", 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196788+0000", 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2120000000000002e-06 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196794+0000", 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7180000000000001e-06 2024-09-13T05:02:41.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196798+0000", 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9709999999999998e-06 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232798+0000", 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035999415 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232871+0000", 2024-09-13T05:02:41.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3837999999999993e-05 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232946+0000", 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30568 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3481600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617952544, 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036337543, 2024-09-13T05:02:41.873 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30568 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196802+0000", 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.395e-06 2024-09-13T05:02:41.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196805+0000", 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2169999999999999e-06 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196808+0000", 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1530000000000001e-06 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232988+0000", 2024-09-13T05:02:41.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036179534999999999 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233068+0000", 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0209000000000004e-05 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233138+0000", 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0308000000000005e-05 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.877 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30569 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196851+0000", 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617902538999999, 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036477633000000002, 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30569 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.878 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196851+0000", 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196851+0000", 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196850+0000", 2024-09-13T05:02:41.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196857+0000", 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1219999999999997e-06 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196865+0000", 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196889+0000", 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4175000000000001e-05 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233178+0000", 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036288735000000003 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233252+0000", 2024-09-13T05:02:41.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4252000000000007e-05 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233328+0000", 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6490999999999997e-05 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30571 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196895+0000", 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617858558, 2024-09-13T05:02:41.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036555702000000002, 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30571 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196895+0000", 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196895+0000", 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196894+0000", 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196897+0000", 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.559e-06 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196904+0000", 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9090000000000003e-06 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196909+0000", 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233364+0000", 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036455682000000003 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233408+0000", 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3819999999999997e-05 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233451+0000", 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2483e-05 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.886 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30573 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617840155, 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036729553999999998, 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.887 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30573 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196915+0000", 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8950000000000001e-06 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196927+0000", 2024-09-13T05:02:41.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2505e-05 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196932+0000", 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7820000000000001e-06 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233490+0000", 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036557665000000003 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233570+0000", 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0298999999999998e-05 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233643+0000", 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2755000000000004e-05 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30575 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196943+0000", 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617810899, 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036806959, 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30575 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196943+0000", 2024-09-13T05:02:41.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196943+0000", 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196942+0000", 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196944+0000", 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.165e-06 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196948+0000", 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2150000000000003e-06 2024-09-13T05:02:41.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196951+0000", 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6429999999999999e-06 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233661+0000", 2024-09-13T05:02:41.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036709498 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233703+0000", 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2527000000000001e-05 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233749+0000", 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6431999999999999e-05 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.896 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30577 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617799371, 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037025295, 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.897 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30577 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196959+0000", 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3340000000000001e-06 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196962+0000", 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2930000000000001e-06 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196965+0000", 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.401e-06 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233783+0000", 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036817772999999998 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233877+0000", 2024-09-13T05:02:41.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4419999999999994e-05 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233979+0000", 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102279 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30579 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617786280000001, 2024-09-13T05:02:41.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037209032000000003, 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30579 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196968+0000", 2024-09-13T05:02:41.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.294e-06 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196971+0000", 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8569999999999999e-06 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196973+0000", 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3870000000000002e-06 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234024+0000", 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037050999000000001 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234104+0000", 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9431999999999994e-05 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234176+0000", 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2292999999999994e-05 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30581 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 4136960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.617777953000001, 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037378281999999999, 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30581 2024-09-13T05:02:41.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:41.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196976+0000", 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.133e-06 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196979+0000", 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0290000000000001e-06 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196982+0000", 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3439999999999999e-06 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234210+0000", 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037228153 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234286+0000", 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5887e-05 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:41.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234354+0000", 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7941999999999994e-05 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:41.910 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:41.911 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:41.912 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:41.910+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:41.995 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:02:42.046 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:42.046 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:42.046 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:42.046 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:42.046 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:42.043+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:42.046 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:42.146 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:42.145+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:42.149 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:42.149 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:42.149 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:42.149 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:42.149 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:42.157 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:42.197 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-13T05:02:42.236 INFO:teuthology.orchestra.run.smithi053.stdout:3446 punch from 0x1350d23 to 0x135933d, (0x861a bytes) 2024-09-13T05:02:42.239 INFO:teuthology.orchestra.run.smithi053.stdout:3447 write 0x96e7521 thru 0x96ee145 (0x6c25 bytes) 2024-09-13T05:02:42.243 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:42.243 INFO:teuthology.orchestra.run.smithi053.stdout:3450 clone 22 order 25 su 524288 sc 9 2024-09-13T05:02:42.247 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:42.245+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:42.264 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:42.264 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:42.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:42.264 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50721 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5654219539999996, 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031376581000000001, 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50721 2024-09-13T05:02:42.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697709+0000", 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9539999999999998e-06 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697714+0000", 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:02:42.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697719+0000", 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6959999999999996e-06 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.725993+0000", 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028273518000000001 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726168+0000", 2024-09-13T05:02:42.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000175632 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729084+0000", 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029156640000000001 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50723 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2138112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697741+0000", 2024-09-13T05:02:42.269 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.565388435, 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028949029000000001, 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50723 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697741+0000", 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697741+0000", 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697740+0000", 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697743+0000", 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5919999999999999e-06 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697749+0000", 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4729999999999997e-06 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697754+0000", 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5179999999999998e-06 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726536+0000", 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028781599000000001 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726611+0000", 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5749999999999998e-05 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726690+0000", 2024-09-13T05:02:42.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8591999999999996e-05 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50724 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5653711999999995, 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029104789999999998, 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50724 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697760+0000", 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4030000000000001e-06 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697766+0000", 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.879e-06 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697770+0000", 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726712+0000", 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028941440999999998 2024-09-13T05:02:42.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726785+0000", 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3492999999999996e-05 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726863+0000", 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7622999999999999e-05 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50725 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2236416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5653550459999996, 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029347541000000001, 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50725 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.279 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:42.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697776+0000", 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.03e-06 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697787+0000", 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1398e-05 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697792+0000", 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726884+0000", 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029092005000000001 2024-09-13T05:02:42.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727020+0000", 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013596999999999999 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727122+0000", 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101674 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50726 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2293760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.565332905, 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029565924, 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50726 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697798+0000", 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1940000000000001e-06 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697804+0000", 2024-09-13T05:02:42.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5809999999999996e-06 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697808+0000", 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3590000000000001e-06 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727193+0000", 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029385402000000001 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727270+0000", 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6571999999999998e-05 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727362+0000", 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2275000000000007e-05 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50727 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2351104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.288 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697812+0000", 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5653175059999995, 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029725557999999999, 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50727 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697812+0000", 2024-09-13T05:02:42.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697812+0000", 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697811+0000", 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697813+0000", 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.86e-06 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697819+0000", 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5169999999999999e-06 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697823+0000", 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4179999999999999e-06 2024-09-13T05:02:42.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727385+0000", 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029562233 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727461+0000", 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5541000000000001e-05 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727537+0000", 2024-09-13T05:02:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6309000000000004e-05 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50728 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2392064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5653018989999996, 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029881569, 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.293 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50728 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:42.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697829+0000", 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.897e-06 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697835+0000", 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6069999999999998e-06 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697839+0000", 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4059999999999998e-06 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727559+0000", 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029720135000000002 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727634+0000", 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4840999999999997e-05 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727709+0000", 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5012999999999995e-05 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.297 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50729 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2449408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697843+0000", 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5652865709999997, 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030040001, 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50729 2024-09-13T05:02:42.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697843+0000", 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697843+0000", 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697842+0000", 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697844+0000", 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9980000000000002e-06 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697850+0000", 2024-09-13T05:02:42.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5849999999999999e-06 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697855+0000", 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6260000000000003e-06 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727730+0000", 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029875432 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.301 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727805+0000", 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5171000000000002e-05 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727883+0000", 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7526000000000003e-05 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50730 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.302 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5652710489999997, 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030203912999999999, 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50730 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.303 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697860+0000", 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9010000000000001e-06 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697865+0000", 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5629999999999999e-06 2024-09-13T05:02:42.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697869+0000", 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727904+0000", 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030034729999999999 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727984+0000", 2024-09-13T05:02:42.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9649999999999998e-05 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728062+0000", 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8369999999999997e-05 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50731 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5652561909999996, 2024-09-13T05:02:42.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030453465999999998, 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50731 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697875+0000", 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9120000000000001e-06 2024-09-13T05:02:42.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697880+0000", 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5339999999999999e-06 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697885+0000", 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.335e-06 2024-09-13T05:02:42.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728084+0000", 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030199756000000001 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728156+0000", 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1732999999999999e-05 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728327+0000", 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017047399999999999 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50732 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2605056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:42.312 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5652370930000004, 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030638446, 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50732 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:42.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:42.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697895+0000", 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3510000000000002e-06 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697901+0000", 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:02:42.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697906+0000", 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000003e-06 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728349+0000", 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030443606000000002 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728445+0000", 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5058999999999999e-05 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728531+0000", 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6069999999999994e-05 2024-09-13T05:02:42.317 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50733 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2646016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.698770+0000", 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5643591949999998, 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029940022, 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.318 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50733 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698770+0000", 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698770+0000", 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698769+0000", 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698773+0000", 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1659999999999996e-06 2024-09-13T05:02:42.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698784+0000", 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1027e-05 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698791+0000", 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.443e-06 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728553+0000", 2024-09-13T05:02:42.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029762365999999998 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728636+0000", 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2792000000000003e-05 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728710+0000", 2024-09-13T05:02:42.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4023000000000006e-05 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50734 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5643323650000003, 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030084214000000001, 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.323 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50734 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698799+0000", 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698805+0000", 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1029999999999998e-06 2024-09-13T05:02:42.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698809+0000", 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5399999999999997e-06 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728732+0000", 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029922338 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728804+0000", 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2305999999999995e-05 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728881+0000", 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7008999999999994e-05 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.327 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50735 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2801664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5643158350000004, 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030249990000000001, 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50735 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698815+0000", 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8199999999999999e-06 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698820+0000", 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4179999999999996e-06 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698825+0000", 2024-09-13T05:02:42.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6199999999999998e-06 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728903+0000", 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030077948 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728984+0000", 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1088999999999996e-05 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729063+0000", 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9375000000000005e-05 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50738 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2957312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700805+0000", 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5623244109999996, 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028972106000000001, 2024-09-13T05:02:42.332 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50738 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.333 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700805+0000", 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700805+0000", 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700804+0000", 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700807+0000", 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3319999999999999e-06 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700814+0000", 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8229999999999997e-06 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700819+0000", 2024-09-13T05:02:42.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.259e-06 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729573+0000", 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028754431 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729670+0000", 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6217000000000004e-05 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729777+0000", 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010744899999999999 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.337 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50739 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3055616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5623057639999995, 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029202947999999999, 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50739 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.338 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700825+0000", 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1349999999999999e-06 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700832+0000", 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7769999999999997e-06 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700838+0000", 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4419999999999997e-06 2024-09-13T05:02:42.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729822+0000", 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028984566999999999 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729911+0000", 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8702999999999994e-05 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730026+0000", 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115648 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50740 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3112960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5622871519999997, 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029383038, 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.342 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50740 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:42.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700844+0000", 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1710000000000001e-06 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700850+0000", 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5740000000000001e-06 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700856+0000", 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2320000000000001e-06 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730055+0000", 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029199583000000001 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730137+0000", 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1476000000000004e-05 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730225+0000", 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8363999999999996e-05 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50741 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5622689779999996, 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029627147999999999, 2024-09-13T05:02:42.347 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50741 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700862+0000", 2024-09-13T05:02:42.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700869+0000", 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5309999999999998e-06 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700874+0000", 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1499999999999998e-06 2024-09-13T05:02:42.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730254+0000", 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029380074999999999 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730335+0000", 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1636000000000002e-05 2024-09-13T05:02:42.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730487+0000", 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015201599999999999 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50742 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3211264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.5622510979999999, 2024-09-13T05:02:42.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029825695999999999, 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50742 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:42.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:42.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700880+0000", 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700886+0000", 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3099999999999997e-06 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700891+0000", 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9130000000000002e-06 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730527+0000", 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029636445000000001 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730615+0000", 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7119000000000004e-05 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730704+0000", 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9303999999999997e-05 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50743 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3268608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4.562233859, 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030066071999999999, 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50743 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700897+0000", 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700903+0000", 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700908+0000", 2024-09-13T05:02:42.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.887e-06 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730767+0000", 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029859041999999999 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730856+0000", 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8986000000000005e-05 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:42.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730961+0000", 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105252 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:42.362 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:42.364 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:42.364 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:42.363+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:42.464 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:42.463+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:42.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:42.505 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:42.505 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:42.505 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:42.506 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:42.514 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:42.565 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:42.563+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:42.615 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:42.615 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:42.616 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:42.616 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:42.624 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:42.631 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:42.631 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:42.647 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:42.648 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:42.663 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:42.664 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:42.666 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:42.665+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:42.680 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:42.680 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:42.697 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:42.697 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:42.714 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:42.714 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:42.731 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:42.731 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:42.767 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:42.765+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:42.769 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:42.769 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:42.778 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:42.868 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:42.867+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:42.884 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:42.885 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:42.893 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:42.969 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:42.969+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:43.044 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:43.045 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:43.054 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:43.054 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone21 from 0xddde10c (overlap 0xddde10c) to 0x428f6b6 2024-09-13T05:02:43.100 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:43.098+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:43.172 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:43.170+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:43.213 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:43.214 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:43.223 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:43.268 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-13T05:02:43.272 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:43.272+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:43.373 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:43.371+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:43.389 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:43.389 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:43.398 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:43.482 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:43.481+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:43.523 INFO:teuthology.orchestra.run.smithi053.stdout:3451 read 0x370eba5 thru 0x371c9a6 (0xde02 bytes) 2024-09-13T05:02:43.534 INFO:teuthology.orchestra.run.smithi053.stdout:3452 trunc from 0xddde10c to 0x5cba68d 2024-09-13T05:02:43.564 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:43.564 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:43.574 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:43.572+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:43.583 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:43.674 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:43.673+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:43.685 INFO:teuthology.orchestra.run.smithi053.stdout:3453 trunc from 0x5cba68d to 0x2a01c64 2024-09-13T05:02:43.700 INFO:teuthology.orchestra.run.smithi053.stdout:3454 write 0x2a19abd thru 0x2a2952b (0xfa6f bytes) 2024-09-13T05:02:43.700 INFO:teuthology.orchestra.run.smithi053.stdout:3456 read 0x77fb7c thru 0x783969 (0x3dee bytes) 2024-09-13T05:02:43.701 INFO:teuthology.orchestra.run.smithi053.stdout:3458 read 0x512bd1 thru 0x513e9e (0x12ce bytes) 2024-09-13T05:02:43.701 INFO:teuthology.orchestra.run.smithi053.stdout:3459 read 0x3e0cb1 thru 0x3e46de (0x3a2e bytes) 2024-09-13T05:02:43.702 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:43.702 INFO:teuthology.orchestra.run.smithi053.stdout:3461 punch from 0x1f40840 to 0x1f4d00e, (0xc7ce bytes) 2024-09-13T05:02:43.719 INFO:teuthology.orchestra.run.smithi053.stdout:3462 write 0xaca86c6 thru 0xacaf59b (0x6ed6 bytes) 2024-09-13T05:02:43.727 INFO:teuthology.orchestra.run.smithi053.stdout:3464 writesame 0xbc534c0 thru 0xbc53cc2 data_size 0x1(0x803 bytes) 2024-09-13T05:02:43.728 INFO:teuthology.orchestra.run.smithi053.stdout:3465 punch from 0x3627c5d to 0x3636abe, (0xee61 bytes) 2024-09-13T05:02:43.742 INFO:teuthology.orchestra.run.smithi053.stdout:3467 trunc from 0xbc53cc3 to 0x111b752 2024-09-13T05:02:43.757 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:43.757 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:43.763 INFO:teuthology.orchestra.run.smithi053.stdout:3468 write 0x6d1b16e thru 0x6d243eb (0x927e bytes) 2024-09-13T05:02:43.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:43.768 INFO:teuthology.orchestra.run.smithi053.stdout:3470 trunc from 0x6d243ec to 0x410210f 2024-09-13T05:02:43.775 INFO:teuthology.orchestra.run.smithi053.stdout:3471 write 0x198f3bf thru 0x1990dbd (0x19ff bytes) 2024-09-13T05:02:43.776 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:43.775+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:43.777 INFO:teuthology.orchestra.run.smithi053.stdout:3472 read 0x917b84 thru 0x91c4a6 (0x4923 bytes) 2024-09-13T05:02:43.784 INFO:teuthology.orchestra.run.smithi053.stdout:3473 writesame 0x5c3ed67 thru 0x5c408bb data_size 0x1(0x1b55 bytes) 2024-09-13T05:02:43.800 INFO:teuthology.orchestra.run.smithi053.stdout:3474 writesame 0xcf2bfe9 thru 0xcf37d4d data_size 0x1(0xbd65 bytes) 2024-09-13T05:02:43.801 INFO:teuthology.orchestra.run.smithi053.stdout:3475 write 0x60d32f3 thru 0x60e0821 (0xd52f bytes) 2024-09-13T05:02:43.804 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:43.804 INFO:teuthology.orchestra.run.smithi053.stdout:3477 writesame 0x3c3a599 thru 0x3c442a5 data_size 0x55(0x9d0d bytes) 2024-09-13T05:02:43.805 INFO:teuthology.orchestra.run.smithi053.stdout:3480 write 0x163bdbb thru 0x163e17f (0x23c5 bytes) 2024-09-13T05:02:43.806 INFO:teuthology.orchestra.run.smithi053.stdout:3481 punch from 0xb56fb44 to 0xb57c426, (0xc8e2 bytes) 2024-09-13T05:02:43.813 INFO:teuthology.orchestra.run.smithi053.stdout:3482 write 0xe8c45e6 thru 0xe8c5495 (0xeb0 bytes) 2024-09-13T05:02:43.815 INFO:teuthology.orchestra.run.smithi053.stdout:3483 write 0x25000b5 thru 0x2506503 (0x644f bytes) 2024-09-13T05:02:43.816 INFO:teuthology.orchestra.run.smithi053.stdout:3484 writesame 0xbf89c34 thru 0xbf97911 data_size 0x11(0xdcde bytes) 2024-09-13T05:02:43.823 INFO:teuthology.orchestra.run.smithi053.stdout:3485 trunc from 0xe8c5496 to 0xaac1705 2024-09-13T05:02:43.830 INFO:teuthology.orchestra.run.smithi053.stdout:3487 write 0x307c4d thru 0x308da4 (0x1158 bytes) 2024-09-13T05:02:43.832 INFO:teuthology.orchestra.run.smithi053.stdout:3488 write 0x9c25aa9 thru 0x9c2d38a (0x78e2 bytes) 2024-09-13T05:02:43.834 INFO:teuthology.orchestra.run.smithi053.stdout:3490 read 0x7216eb thru 0x724cdf (0x35f5 bytes) 2024-09-13T05:02:43.834 INFO:teuthology.orchestra.run.smithi053.stdout:3491 punch from 0x1314b68 to 0x1317c39, (0x30d1 bytes) 2024-09-13T05:02:43.840 INFO:teuthology.orchestra.run.smithi053.stdout:3492 write 0xb0e15dc thru 0xb0e7a6e (0x6493 bytes) 2024-09-13T05:02:43.841 INFO:teuthology.orchestra.run.smithi053.stdout:3493 write 0x2d63834 thru 0x2d6c3e0 (0x8bad bytes) 2024-09-13T05:02:43.842 INFO:teuthology.orchestra.run.smithi053.stdout:3494 read 0x37f7c33 thru 0x3801758 (0x9b26 bytes) 2024-09-13T05:02:43.843 INFO:teuthology.orchestra.run.smithi053.stdout:3495 write 0x4139a4b thru 0x4140034 (0x65ea bytes) 2024-09-13T05:02:43.850 INFO:teuthology.orchestra.run.smithi053.stdout:3496 trunc from 0xb0e7a6f to 0x671a8f2 2024-09-13T05:02:43.863 INFO:teuthology.orchestra.run.smithi053.stdout:3497 trunc from 0x671a8f2 to 0xd1206d 2024-09-13T05:02:43.877 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:43.875+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:43.882 INFO:teuthology.orchestra.run.smithi053.stdout:3498 write 0x6b80e5b thru 0x6b85c7d (0x4e23 bytes) 2024-09-13T05:02:43.892 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:43.892 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:43.893 INFO:teuthology.orchestra.run.smithi053.stdout:3500 write 0x9b5d222 thru 0x9b5d96e (0x74d bytes) 2024-09-13T05:02:43.901 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:43.903 INFO:teuthology.orchestra.run.smithi053.stdout:3501 write 0xc43b385 thru 0xc44147a (0x60f6 bytes) 2024-09-13T05:02:43.905 INFO:teuthology.orchestra.run.smithi053.stdout:3502 write 0x7a49816 thru 0x7a4f0e2 (0x58cd bytes) 2024-09-13T05:02:43.906 INFO:teuthology.orchestra.run.smithi053.stdout:3503 writesame 0x8c832e5 thru 0x8c92aaf data_size 0x5(0xf7cb bytes) 2024-09-13T05:02:43.917 INFO:teuthology.orchestra.run.smithi053.stdout:3504 write 0xc746d4f thru 0xc754dd7 (0xe089 bytes) 2024-09-13T05:02:43.919 INFO:teuthology.orchestra.run.smithi053.stdout:3505 write 0x69f25e3 thru 0x69fac80 (0x869e bytes) 2024-09-13T05:02:43.921 INFO:teuthology.orchestra.run.smithi053.stdout:3506 writesame 0x25dce5f thru 0x25df589 data_size 0x1(0x272b bytes) 2024-09-13T05:02:43.927 INFO:teuthology.orchestra.run.smithi053.stdout:3507 read 0x7b691e1 thru 0x7b6ceef (0x3d0f bytes) 2024-09-13T05:02:43.927 INFO:teuthology.orchestra.run.smithi053.stdout:3508 read 0x16dc2b7 thru 0x16e725b (0xafa5 bytes) 2024-09-13T05:02:43.927 INFO:teuthology.orchestra.run.smithi053.stdout:3509 writesame 0x8e5d77a thru 0x8e66bf8 data_size 0x1(0x947f bytes) 2024-09-13T05:02:43.945 INFO:teuthology.orchestra.run.smithi053.stdout:3511 read 0x850db29 thru 0x851c428 (0xe900 bytes) 2024-09-13T05:02:43.946 INFO:teuthology.orchestra.run.smithi053.stdout:3512 writesame 0xc378df7 thru 0xc387d20 data_size 0x79(0xef2a bytes) 2024-09-13T05:02:43.947 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:43.948 INFO:teuthology.orchestra.run.smithi053.stdout:3514 write 0x85a6ed8 thru 0x85b6490 (0xf5b9 bytes) 2024-09-13T05:02:43.959 INFO:teuthology.orchestra.run.smithi053.stdout:3515 trunc from 0xc754dd8 to 0x58a9875 2024-09-13T05:02:43.977 INFO:teuthology.orchestra.run.smithi053.stdout:3517 writesame 0xa1ef4fd thru 0xa1f7681 data_size 0x1(0x8185 bytes) 2024-09-13T05:02:43.978 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:43.977+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:43.978 INFO:teuthology.orchestra.run.smithi053.stdout:3518 read 0x143cdf9 thru 0x1440999 (0x3ba1 bytes) 2024-09-13T05:02:43.984 INFO:teuthology.orchestra.run.smithi053.stdout:3519 write 0xaf88c84 thru 0xaf8dcdb (0x5058 bytes) 2024-09-13T05:02:43.986 INFO:teuthology.orchestra.run.smithi053.stdout:3521 read 0xab1db05 thru 0xab2d492 (0xf98e bytes) 2024-09-13T05:02:43.986 INFO:teuthology.orchestra.run.smithi053.stdout:3522 writesame 0x4c23923 thru 0x4c29f06 data_size 0x2(0x65e4 bytes) 2024-09-13T05:02:43.993 INFO:teuthology.orchestra.run.smithi053.stdout:3523 punch from 0xa290a22 to 0xa292497, (0x1a75 bytes) 2024-09-13T05:02:43.993 INFO:teuthology.orchestra.run.smithi053.stdout:3526 read 0x26cd25c thru 0x26d5276 (0x801b bytes) 2024-09-13T05:02:43.993 INFO:teuthology.orchestra.run.smithi053.stdout:3527 write 0xa6569a8 thru 0xa662c60 (0xc2b9 bytes) 2024-09-13T05:02:43.995 INFO:teuthology.orchestra.run.smithi053.stdout:3528 read 0x3ee1033 thru 0x3eea792 (0x9760 bytes) 2024-09-13T05:02:43.995 INFO:teuthology.orchestra.run.smithi053.stdout:3531 write 0x97590d9 thru 0x9766434 (0xd35c bytes) 2024-09-13T05:02:43.997 INFO:teuthology.orchestra.run.smithi053.stdout:3532 punch from 0x3e51155 to 0x3e530a6, (0x1f51 bytes) 2024-09-13T05:02:43.997 INFO:teuthology.orchestra.run.smithi053.stdout:3533 read 0x69faab5 thru 0x6a06c27 (0xc173 bytes) 2024-09-13T05:02:43.997 INFO:teuthology.orchestra.run.smithi053.stdout:3534 writesame 0x4e84629 thru 0x4e8e9de data_size 0xb(0xa3b6 bytes) 2024-09-13T05:02:44.000 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:44.000 INFO:teuthology.orchestra.run.smithi053.stdout:3537 read 0x60040c8 thru 0x6011a59 (0xd992 bytes) 2024-09-13T05:02:44.000 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:44.001 INFO:teuthology.orchestra.run.smithi053.stdout:3542 write 0x8b191d thru 0x8bfc2d (0xe311 bytes) 2024-09-13T05:02:44.002 INFO:teuthology.orchestra.run.smithi053.stdout:3543 read 0x35b5ce0 thru 0x35b8af7 (0x2e18 bytes) 2024-09-13T05:02:44.003 INFO:teuthology.orchestra.run.smithi053.stdout:3544 write 0x27fc2cf thru 0x280acc3 (0xe9f5 bytes) 2024-09-13T05:02:44.011 INFO:teuthology.orchestra.run.smithi053.stdout:3545 write 0xc0a5dc7 thru 0xc0b5ab5 (0xfcef bytes) 2024-09-13T05:02:44.012 INFO:teuthology.orchestra.run.smithi053.stdout:3547 punch from 0x3b9b1d5 to 0x3ba1894, (0x66bf bytes) 2024-09-13T05:02:44.012 INFO:teuthology.orchestra.run.smithi053.stdout:3548 read 0x4921893 thru 0x4924104 (0x2872 bytes) 2024-09-13T05:02:44.014 INFO:teuthology.orchestra.run.smithi053.stdout:3549 trunc from 0xc0b5ab6 to 0xd3ce0cb 2024-09-13T05:02:44.019 INFO:teuthology.orchestra.run.smithi053.stdout:3552 punch from 0x543660e to 0x54369cd, (0x3bf bytes) 2024-09-13T05:02:44.020 INFO:teuthology.orchestra.run.smithi053.stdout:3553 trunc from 0xd3ce0cb to 0xc2e8a66 2024-09-13T05:02:44.027 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.027 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.027 INFO:teuthology.orchestra.run.smithi053.stdout:3554 read 0xc08e83b thru 0xc09dad6 (0xf29c bytes) 2024-09-13T05:02:44.035 INFO:teuthology.orchestra.run.smithi053.stdout:3556 write 0xe252b93 thru 0xe2591b3 (0x6621 bytes) 2024-09-13T05:02:44.037 INFO:teuthology.orchestra.run.smithi053.stdout:3557 read 0x46724f3 thru 0x467ae22 (0x8930 bytes) 2024-09-13T05:02:44.037 INFO:teuthology.orchestra.run.smithi053.stdout:3558 read 0x784e77a thru 0x784fac9 (0x1350 bytes) 2024-09-13T05:02:44.037 INFO:teuthology.orchestra.run.smithi053.stdout:3559 read 0x826fbc1 thru 0x8276004 (0x6444 bytes) 2024-09-13T05:02:44.038 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:44.039 INFO:teuthology.orchestra.run.smithi053.stdout:3561 clone 23 order 19 su 65536 sc 10 2024-09-13T05:02:44.047 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:02:44.079 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:44.078+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:44.180 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:44.179+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:44.187 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.187 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.199 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:44.280 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:44.280+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:44.306 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.306 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:44.381 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:44.380+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:44.421 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.422 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.430 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:44.482 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:44.481+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:44.538 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.538 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:44.587 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:44.586+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:44.651 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.651 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.660 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:44.703 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:44.703+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:44.743 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:44.761 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.761 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.770 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:44.811 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:44.810+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:44.867 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:44.868 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:44.884 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:44.884 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:44.886 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:44.886+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:44.899 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:44.900 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:44.913 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:44.913 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:44.915 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:44.915 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:44.922 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:44.931 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:44.932 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:44.948 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:44.948 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:44.965 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:44.965 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:44.981 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:44.981 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:45.004 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:02:45.005 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:45.005+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:45.069 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.069 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:45.069 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:45.069 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.070 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone22 from 0xe2591b4 (overlap 0xd1206d) to 0x0 2024-09-13T05:02:45.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:45.103 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:45.103 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:45.103 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-13T05:02:45.103 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:02:45.103 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:02:45.103 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-13T05:02:45.121 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:45.120+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:45.132 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-13T05:02:45.190 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:45.188+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:45.219 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.219 INFO:teuthology.orchestra.run.smithi053.stdout:3564 write 0x1038882 thru 0x1038988 (0x107 bytes) 2024-09-13T05:02:45.227 INFO:teuthology.orchestra.run.smithi053.stdout:3565 writesame 0x3f8b727 thru 0x3f973f1 data_size 0x1(0xbccb bytes) 2024-09-13T05:02:45.228 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T05:02:45.228 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.229 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:45.229 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.241 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:45.259 INFO:teuthology.orchestra.run.smithi053.stdout:3567 trunc from 0xe2591b4 to 0x445e62f 2024-09-13T05:02:45.278 INFO:teuthology.orchestra.run.smithi053.stdout:3568 trunc from 0x445e62f to 0x589d70e 2024-09-13T05:02:45.299 INFO:teuthology.orchestra.run.smithi053.stdout:3569 write 0xcb6ca8a thru 0xcb77e3b (0xb3b2 bytes) 2024-09-13T05:02:45.303 INFO:teuthology.orchestra.run.smithi053.stdout:3570 punch from 0x27774f4 to 0x277cc0d, (0x5719 bytes) 2024-09-13T05:02:45.303 INFO:teuthology.orchestra.run.smithi053.stdout:3571 read 0x1989feb thru 0x198cc89 (0x2c9f bytes) 2024-09-13T05:02:45.303 INFO:teuthology.orchestra.run.smithi053.stdout:3572 write 0x8257153 thru 0x8266630 (0xf4de bytes) 2024-09-13T05:02:45.307 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.307 INFO:teuthology.orchestra.run.smithi053.stdout:3575 writesame 0x80f96a2 thru 0x8107db5 data_size 0x2e(0xe714 bytes) 2024-09-13T05:02:45.312 INFO:teuthology.orchestra.run.smithi053.stdout:3577 write 0x10e8a3c thru 0x10f2824 (0x9de9 bytes) 2024-09-13T05:02:45.316 INFO:teuthology.orchestra.run.smithi053.stdout:3578 writesame 0x1af9183 thru 0x1b01e0e data_size 0x46(0x8c8c bytes) 2024-09-13T05:02:45.321 INFO:teuthology.orchestra.run.smithi053.stdout:3579 write 0x444a931 thru 0x44547f0 (0x9ec0 bytes) 2024-09-13T05:02:45.323 INFO:teuthology.orchestra.run.smithi053.stdout:3580 write 0x9765247 thru 0x976cc2e (0x79e8 bytes) 2024-09-13T05:02:45.324 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:45.324+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:45.336 INFO:teuthology.orchestra.run.smithi053.stdout:3582 trunc from 0xcb77e3c to 0x3cd437e 2024-09-13T05:02:45.363 INFO:teuthology.orchestra.run.smithi053.stdout:3583 read 0x24f6588 thru 0x2504594 (0xe00d bytes) 2024-09-13T05:02:45.363 INFO:teuthology.orchestra.run.smithi053.stdout:3584 read 0x27f7786 thru 0x2801023 (0x989e bytes) 2024-09-13T05:02:45.376 INFO:teuthology.orchestra.run.smithi053.stdout:3585 trunc from 0x3cd437e to 0xe8d0b52 2024-09-13T05:02:45.382 INFO:teuthology.orchestra.run.smithi053.stdout:3586 writesame 0xca2d13f thru 0xca36b9f data_size 0x1(0x9a61 bytes) 2024-09-13T05:02:45.392 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:45.391+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30534 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.397 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194558+0000", 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.201127811999999, 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035531742999999998, 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30534 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.398 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194558+0000", 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194558+0000", 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194557+0000", 2024-09-13T05:02:45.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194560+0000", 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2689999999999998e-06 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194566+0000", 2024-09-13T05:02:45.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3289999999999999e-06 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194576+0000", 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5310000000000007e-06 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.229910+0000", 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035333950000000003 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.229995+0000", 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5227000000000002e-05 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230090+0000", 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4876000000000004e-05 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30535 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.402 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194581+0000", 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.201105016, 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035632084000000001, 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30535 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194581+0000", 2024-09-13T05:02:45.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194581+0000", 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194580+0000", 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194582+0000", 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.154e-06 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194589+0000", 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3589999999999996e-06 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194595+0000", 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6779999999999997e-06 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230117+0000", 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035522977999999997 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230158+0000", 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0664999999999998e-05 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230213+0000", 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.465e-05 2024-09-13T05:02:45.406 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30536 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.201086711999999, 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035829891000000003, 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.407 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30536 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194599+0000", 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194601+0000", 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0389999999999999e-06 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194607+0000", 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194612+0000", 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1610000000000002e-06 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230248+0000", 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035635714999999998 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230331+0000", 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3089000000000004e-05 2024-09-13T05:02:45.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230429+0000", 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8107000000000001e-05 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30537 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2138112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.201069384, 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036017699, 2024-09-13T05:02:45.411 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30537 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194616+0000", 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194618+0000", 2024-09-13T05:02:45.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9889999999999999e-06 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194624+0000", 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0109999999999997e-06 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194636+0000", 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1738e-05 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230464+0000", 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035828477999999997 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230543+0000", 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8838000000000006e-05 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230634+0000", 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.1013000000000004e-05 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.415 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30540 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2236416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194649+0000", 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.201037075999999, 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036213493999999999, 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.416 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30540 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194649+0000", 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194649+0000", 2024-09-13T05:02:45.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194648+0000", 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194650+0000", 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3029999999999998e-06 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194663+0000", 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3012e-05 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194671+0000", 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230675+0000", 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036004622 2024-09-13T05:02:45.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230761+0000", 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5778000000000004e-05 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230862+0000", 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000100936 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30542 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2334720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.194676+0000", 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.201010139000001, 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036384622999999998, 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30542 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194676+0000", 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194676+0000", 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194675+0000", 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194677+0000", 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2299999999999998e-06 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194684+0000", 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3600000000000001e-06 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.194688+0000", 2024-09-13T05:02:45.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6190000000000002e-06 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230881+0000", 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036192584 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.230961+0000", 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0389000000000006e-05 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.231060+0000", 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8808000000000007e-05 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30559 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 2924544~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196700+0000", 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198985928999999, 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035438492000000002, 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30559 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196700+0000", 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196700+0000", 2024-09-13T05:02:45.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196699+0000", 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196703+0000", 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.709e-06 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196714+0000", 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1080999999999999e-05 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196721+0000", 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8140000000000003e-06 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232062+0000", 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035340871000000003 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232099+0000", 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.68e-05 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232138+0000", 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9743999999999998e-05 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30561 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3022848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198959416000001, 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035553504999999999, 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30561 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.430 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196726+0000", 2024-09-13T05:02:45.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196729+0000", 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7240000000000001e-06 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196735+0000", 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4579999999999998e-06 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196740+0000", 2024-09-13T05:02:45.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9180000000000002e-06 2024-09-13T05:02:45.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232165+0000", 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035424663000000002 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232219+0000", 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4317000000000002e-05 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232280+0000", 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0871e-05 2024-09-13T05:02:45.434 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30562 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3121152~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196745+0000", 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198941088000002, 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035759104999999999, 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.435 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30562 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196745+0000", 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196745+0000", 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196744+0000", 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196746+0000", 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5659999999999999e-06 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196750+0000", 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.636e-06 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196753+0000", 2024-09-13T05:02:45.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.021e-06 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232325+0000", 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035571924999999997 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232409+0000", 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4376999999999995e-05 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232504+0000", 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4843e-05 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30564 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3153920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198930766, 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035867128999999998, 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.440 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30564 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:45.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196755+0000", 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196756+0000", 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2890000000000001e-06 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196759+0000", 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.932e-06 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196767+0000", 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1110000000000008e-06 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232537+0000", 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035769586999999999 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232582+0000", 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5358000000000002e-05 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232622+0000", 2024-09-13T05:02:45.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0057e-05 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30565 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3252224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198915669000002, 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035982617000000001, 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.445 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30565 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:45.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196770+0000", 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196772+0000", 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0389999999999999e-06 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196778+0000", 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2210000000000002e-06 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196782+0000", 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4070000000000003e-06 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232647+0000", 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035864723000000001 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232707+0000", 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9416999999999997e-05 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232753+0000", 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6156999999999999e-05 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30566 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3350528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196787+0000", 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198899113, 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036159697999999997, 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30566 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196787+0000", 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196787+0000", 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196786+0000", 2024-09-13T05:02:45.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196788+0000", 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2120000000000002e-06 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196794+0000", 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7180000000000001e-06 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196798+0000", 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9709999999999998e-06 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232798+0000", 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035999415 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232871+0000", 2024-09-13T05:02:45.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3837999999999993e-05 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232946+0000", 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4939999999999997e-05 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30568 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3481600~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:45.454 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198884886999998, 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036337543, 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30568 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196801+0000", 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196802+0000", 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.395e-06 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196805+0000", 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2169999999999999e-06 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196808+0000", 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1530000000000001e-06 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.232988+0000", 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036179534999999999 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233068+0000", 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0209000000000004e-05 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233138+0000", 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0308000000000005e-05 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30569 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3579904~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196851+0000", 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198834882, 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036477633000000002, 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.459 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30569 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196851+0000", 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.460 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196851+0000", 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196850+0000", 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196857+0000", 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1219999999999997e-06 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196865+0000", 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196889+0000", 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4175000000000001e-05 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233178+0000", 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036288735000000003 2024-09-13T05:02:45.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233252+0000", 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4252000000000007e-05 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233328+0000", 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6490999999999997e-05 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30571 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3710976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196895+0000", 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198790900999999, 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036555702000000002, 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30571 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196895+0000", 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196895+0000", 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196894+0000", 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196897+0000", 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.559e-06 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196904+0000", 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9090000000000003e-06 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196909+0000", 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233364+0000", 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036455682000000003 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233408+0000", 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3819999999999997e-05 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233451+0000", 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2483e-05 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30573 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3809280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198772498, 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036729553999999998, 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.468 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30573 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196913+0000", 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196915+0000", 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8950000000000001e-06 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196927+0000", 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2505e-05 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196932+0000", 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7820000000000001e-06 2024-09-13T05:02:45.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233490+0000", 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036557665000000003 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233570+0000", 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0298999999999998e-05 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233643+0000", 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2755000000000004e-05 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30575 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3907584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196943+0000", 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198743241999999, 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036806959, 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.473 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30575 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196943+0000", 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196943+0000", 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196942+0000", 2024-09-13T05:02:45.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196944+0000", 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.165e-06 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196948+0000", 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2150000000000003e-06 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196951+0000", 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6429999999999999e-06 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233661+0000", 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036709498 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233703+0000", 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2527000000000001e-05 2024-09-13T05:02:45.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233749+0000", 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6431999999999999e-05 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30577 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 3940352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198731714000001, 2024-09-13T05:02:45.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037025295, 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30577 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:45.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.481 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196954+0000", 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196959+0000", 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3340000000000001e-06 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196962+0000", 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2930000000000001e-06 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.482 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196965+0000", 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.401e-06 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233783+0000", 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036817772999999998 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233877+0000", 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.4419999999999994e-05 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.483 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.233979+0000", 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000102279 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30579 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 4038656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198718623000001, 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037209032000000003, 2024-09-13T05:02:45.484 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30579 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.485 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196967+0000", 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196968+0000", 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.294e-06 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.486 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196971+0000", 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8569999999999999e-06 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196973+0000", 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3870000000000002e-06 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234024+0000", 2024-09-13T05:02:45.487 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037050999000000001 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234104+0000", 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9431999999999994e-05 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234176+0000", 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2292999999999994e-05 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.488 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:30581 3.1c 3:390d2c80:::rbd_data.117c7aa26b99.0000000000000007:d [read 4136960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 18.198710296000002, 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037378281999999999, 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 30581 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.489 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196975+0000", 2024-09-13T05:02:45.490 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196976+0000", 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.133e-06 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196979+0000", 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0290000000000001e-06 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.491 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.196982+0000", 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3439999999999999e-06 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234210+0000", 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037228153 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234286+0000", 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5887e-05 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.492 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.234354+0000", 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7941999999999994e-05 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.493 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.495 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:45.496 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:45.494+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3587 trunc from 0xe8d0b52 to 0x3456fde 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3589 read 0x22c34e8 thru 0x22c85fd (0x5116 bytes) 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3590 read 0xfa6de4 thru 0xfb07d7 (0x99f4 bytes) 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3591 write 0xbb1a60d thru 0xbb20b88 (0x657c bytes) 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3592 writesame 0xac1074f thru 0xac169fe data_size 0x8(0x62b0 bytes) 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3595 writesame 0x1d575fc thru 0x1d64811 data_size 0x1(0xd216 bytes) 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3597 read 0x9fea2c4 thru 0x9fee34f (0x408c bytes) 2024-09-13T05:02:45.497 INFO:teuthology.orchestra.run.smithi053.stdout:3598 punch from 0x8efc907 to 0x8f07345, (0xaa3e bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3599 read 0xa8f4b95 thru 0xa8fad33 (0x619f bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3601 writesame 0xb18f9b2 thru 0xb19bdfe data_size 0x7(0xc44d bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3604 read 0xaa96492 thru 0xaaa292a (0xc499 bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3606 punch from 0x1f2eaae to 0x1f32ce8, (0x423a bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3607 read 0x5b00dc3 thru 0x5b0a49c (0x96da bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3608 read 0x253c99 thru 0x25db7a (0x9ee2 bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3609 write 0xad304b1 thru 0xad36a86 (0x65d6 bytes) 2024-09-13T05:02:45.498 INFO:teuthology.orchestra.run.smithi053.stdout:3610 trunc from 0xbb20b89 to 0x85de897 2024-09-13T05:02:45.505 INFO:teuthology.orchestra.run.smithi053.stdout:3611 punch from 0x3304b26 to 0x330731e, (0x27f8 bytes) 2024-09-13T05:02:45.513 INFO:teuthology.orchestra.run.smithi053.stdout:3613 trunc from 0x85de897 to 0xe67a60 2024-09-13T05:02:45.546 INFO:teuthology.orchestra.run.smithi053.stdout:3614 write 0x27f3bd8 thru 0x27f800b (0x4434 bytes) 2024-09-13T05:02:45.550 INFO:teuthology.orchestra.run.smithi053.stdout:3615 writesame 0x18995d1 thru 0x189d9cb data_size 0x1(0x43fb bytes) 2024-09-13T05:02:45.562 INFO:teuthology.orchestra.run.smithi053.stdout:3616 read 0x7e2412 thru 0x7e6ee7 (0x4ad6 bytes) 2024-09-13T05:02:45.578 INFO:teuthology.orchestra.run.smithi053.stdout:3618 write 0xa18c3f7 thru 0xa1916f7 (0x5301 bytes) 2024-09-13T05:02:45.582 INFO:teuthology.orchestra.run.smithi053.stdout:3619 punch from 0x1240a98 to 0x12440f6, (0x365e bytes) 2024-09-13T05:02:45.582 INFO:teuthology.orchestra.run.smithi053.stdout:3620 writesame 0x8543a5c thru 0x854fae9 data_size 0x31(0xc08e bytes) 2024-09-13T05:02:45.584 INFO:teuthology.orchestra.run.smithi053.stdout:3622 read 0x352826e thru 0x352f3a9 (0x713c bytes) 2024-09-13T05:02:45.594 INFO:teuthology.orchestra.run.smithi053.stdout:3623 write 0xd8c2193 thru 0xd8c6198 (0x4006 bytes) 2024-09-13T05:02:45.595 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:45.593+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:45.608 INFO:teuthology.orchestra.run.smithi053.stdout:3624 trunc from 0xd8c6199 to 0x3b6c9e7 2024-09-13T05:02:45.633 INFO:teuthology.orchestra.run.smithi053.stdout:3625 write 0xaa16bd4 thru 0xaa1bc77 (0x50a4 bytes) 2024-09-13T05:02:45.644 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.644 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:45.645 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.647 INFO:teuthology.orchestra.run.smithi053.stdout:3626 trunc from 0xaa1bc78 to 0x1187afe 2024-09-13T05:02:45.654 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:45.657 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.657 INFO:teuthology.orchestra.run.smithi053.stdout:3628 punch from 0xf80837 to 0xf817af, (0xf78 bytes) 2024-09-13T05:02:45.669 INFO:teuthology.orchestra.run.smithi053.stdout:3629 write 0x6cb59e9 thru 0x6cb80df (0x26f7 bytes) 2024-09-13T05:02:45.672 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.672 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.684 INFO:teuthology.orchestra.run.smithi053.stdout:3636 write 0xbf80172 thru 0xbf84cc7 (0x4b56 bytes) 2024-09-13T05:02:45.688 INFO:teuthology.orchestra.run.smithi053.stdout:3637 trunc from 0xbf84cc8 to 0xd12f742 2024-09-13T05:02:45.696 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:45.694+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:45.698 INFO:teuthology.orchestra.run.smithi053.stdout:3638 trunc from 0xd12f742 to 0x8bb1323 2024-09-13T05:02:45.709 INFO:teuthology.orchestra.run.smithi053.stdout:3639 write 0x7273ca6 thru 0x7277883 (0x3bde bytes) 2024-09-13T05:02:45.712 INFO:teuthology.orchestra.run.smithi053.stdout:3640 write 0x1994fa9 thru 0x19a2697 (0xd6ef bytes) 2024-09-13T05:02:45.716 INFO:teuthology.orchestra.run.smithi053.stdout:3642 write 0x6147f64 thru 0x6152863 (0xa900 bytes) 2024-09-13T05:02:45.720 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.720 INFO:teuthology.orchestra.run.smithi053.stdout:3644 read 0x582fa7b thru 0x583bc6f (0xc1f5 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3646 read 0x371631b thru 0x37244ce (0xe1b4 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3647 punch from 0x27b3799 to 0x27bfa2f, (0xc296 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3648 read 0x4479a2a thru 0x4484f21 (0xb4f8 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3650 read 0x5c034cf thru 0x5c0dbc0 (0xa6f2 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3651 punch from 0x9d13c3 to 0x9d3b2a, (0x2767 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3652 punch from 0x26713c0 to 0x267b42a, (0xa06a bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3654 read 0x80b109b thru 0x80b1772 (0x6d8 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3655 punch from 0x1d82c57 to 0x1d87d26, (0x50cf bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3656 read 0x59e490a thru 0x59ec182 (0x7879 bytes) 2024-09-13T05:02:45.721 INFO:teuthology.orchestra.run.smithi053.stdout:3657 write 0x66f1d2d thru 0x66fd362 (0xb636 bytes) 2024-09-13T05:02:45.723 INFO:teuthology.orchestra.run.smithi053.stdout:3659 writesame 0x6d447b9 thru 0x6d49d80 data_size 0x78(0x55c8 bytes) 2024-09-13T05:02:45.726 INFO:teuthology.orchestra.run.smithi053.stdout:3660 write 0xa1b953 thru 0xa20452 (0x4b00 bytes) 2024-09-13T05:02:45.728 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.728 INFO:teuthology.orchestra.run.smithi053.stdout:3663 punch from 0x3622b65 to 0x363215b, (0xf5f6 bytes) 2024-09-13T05:02:45.728 INFO:teuthology.orchestra.run.smithi053.stdout:3664 write 0x8a9ff1c thru 0x8aa89b2 (0x8a97 bytes) 2024-09-13T05:02:45.738 INFO:teuthology.orchestra.run.smithi053.stdout:3666 writesame 0x8e24427 thru 0x8e293fc data_size 0xb(0x4fd6 bytes) 2024-09-13T05:02:45.738 INFO:teuthology.orchestra.run.smithi053.stdout:3667 read 0x3f6e7c2 thru 0x3f7b0cd (0xc90c bytes) 2024-09-13T05:02:45.738 INFO:teuthology.orchestra.run.smithi053.stdout:3669 write 0x3991e47 thru 0x399a926 (0x8ae0 bytes) 2024-09-13T05:02:45.741 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.741 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.741 INFO:teuthology.orchestra.run.smithi053.stdout:3674 punch from 0x2045b3e to 0x2049f09, (0x43cb bytes) 2024-09-13T05:02:45.748 INFO:teuthology.orchestra.run.smithi053.stdout:3675 writesame 0x9735069 thru 0x9735d83 data_size 0xb(0xd1b bytes) 2024-09-13T05:02:45.759 INFO:teuthology.orchestra.run.smithi053.stdout:3676 write 0xe16c0ea thru 0xe16ead2 (0x29e9 bytes) 2024-09-13T05:02:45.763 INFO:teuthology.orchestra.run.smithi053.stdout:3678 read 0x7453dda thru 0x746383c (0xfa63 bytes) 2024-09-13T05:02:45.763 INFO:teuthology.orchestra.run.smithi053.stdout:3679 read 0x78c813b thru 0x78d4bdb (0xcaa1 bytes) 2024-09-13T05:02:45.763 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.763 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:45.764 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:45.764 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:45.764 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.773 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:45.773 INFO:teuthology.orchestra.run.smithi053.stdout:3680 trunc from 0xe16ead3 to 0x552b085 2024-09-13T05:02:45.786 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.797 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:45.795+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:45.802 INFO:teuthology.orchestra.run.smithi053.stdout:3682 writesame 0xe0b39ae thru 0xe0b6e6c data_size 0x7(0x34bf bytes) 2024-09-13T05:02:45.802 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.810 INFO:teuthology.orchestra.run.smithi053.stdout:3685 trunc from 0xe0b6e6d to 0x6bb4346 2024-09-13T05:02:45.816 INFO:teuthology.orchestra.run.smithi053.stdout:3686 write 0x13d65b1 thru 0x13dc616 (0x6066 bytes) 2024-09-13T05:02:45.820 INFO:teuthology.orchestra.run.smithi053.stdout:3687 punch from 0x2fa5fa3 to 0x2fafa1a, (0x9a77 bytes) 2024-09-13T05:02:45.827 INFO:teuthology.orchestra.run.smithi053.stdout:3688 writesame 0x7c8753f thru 0x7c92752 data_size 0x3a(0xb214 bytes) 2024-09-13T05:02:45.836 INFO:teuthology.orchestra.run.smithi053.stdout:3690 write 0xa37133f thru 0xa37ef89 (0xdc4b bytes) 2024-09-13T05:02:45.851 INFO:teuthology.orchestra.run.smithi053.stdout:3691 write 0xdf38e36 thru 0xdf3fd53 (0x6f1e bytes) 2024-09-13T05:02:45.855 INFO:teuthology.orchestra.run.smithi053.stdout:3693 read 0x6998aa thru 0x6a53c5 (0xbb1c bytes) 2024-09-13T05:02:45.855 INFO:teuthology.orchestra.run.smithi053.stdout:3694 writesame 0x54ba127 thru 0x54c06b3 data_size 0x1(0x658d bytes) 2024-09-13T05:02:45.869 INFO:teuthology.orchestra.run.smithi053.stdout:3695 punch from 0x350c50e to 0x3513655, (0x7147 bytes) 2024-09-13T05:02:45.869 INFO:teuthology.orchestra.run.smithi053.stdout:3696 write 0x7b07dcf thru 0x7b179eb (0xfc1d bytes) 2024-09-13T05:02:45.874 INFO:teuthology.orchestra.run.smithi053.stdout:3697 read 0x4175075 thru 0x417e298 (0x9224 bytes) 2024-09-13T05:02:45.874 INFO:teuthology.orchestra.run.smithi053.stdout:3698 punch from 0x164eb3a to 0x1651f29, (0x33ef bytes) 2024-09-13T05:02:45.874 INFO:teuthology.orchestra.run.smithi053.stdout:3699 write 0x99d4f26 thru 0x99ddbbc (0x8c97 bytes) 2024-09-13T05:02:45.877 INFO:teuthology.orchestra.run.smithi053.stdout:3701 read 0xa43bdbd thru 0xa4478b4 (0xbaf8 bytes) 2024-09-13T05:02:45.877 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:45.880 INFO:teuthology.orchestra.run.smithi053.stdout:3703 trunc from 0xdf3fd54 to 0xaf38f4c 2024-09-13T05:02:45.884 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:45.884 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50721 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.184349525, 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.031376581000000001, 2024-09-13T05:02:45.885 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50721 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:45.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697707+0000", 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697709+0000", 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9539999999999998e-06 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697714+0000", 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.888 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697719+0000", 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6959999999999996e-06 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.725993+0000", 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028273518000000001 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726168+0000", 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000175632 2024-09-13T05:02:45.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729084+0000", 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029156640000000001 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50723 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2138112~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697741+0000", 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1843160059999995, 2024-09-13T05:02:45.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028949029000000001, 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50723 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697741+0000", 2024-09-13T05:02:45.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697741+0000", 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697740+0000", 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697743+0000", 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5919999999999999e-06 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697749+0000", 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4729999999999997e-06 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697754+0000", 2024-09-13T05:02:45.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5179999999999998e-06 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726536+0000", 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028781599000000001 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726611+0000", 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5749999999999998e-05 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726690+0000", 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8591999999999996e-05 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.895 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50724 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2195456~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1842987709999999, 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029104789999999998, 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50724 2024-09-13T05:02:45.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697758+0000", 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697760+0000", 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4030000000000001e-06 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697766+0000", 2024-09-13T05:02:45.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.879e-06 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697770+0000", 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726712+0000", 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028941440999999998 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726785+0000", 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3492999999999996e-05 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726863+0000", 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7622999999999999e-05 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50725 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2236416~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:45.900 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1842826169999991, 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029347541000000001, 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50725 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:45.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697774+0000", 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.902 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697776+0000", 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.03e-06 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697787+0000", 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1398e-05 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697792+0000", 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.726884+0000", 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029092005000000001 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727020+0000", 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00013596999999999999 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727122+0000", 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000101674 2024-09-13T05:02:45.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50726 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2293760~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1842604760000004, 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029565924, 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.905 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50726 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697796+0000", 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697798+0000", 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1940000000000001e-06 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697804+0000", 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5809999999999996e-06 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697808+0000", 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3590000000000001e-06 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727193+0000", 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029385402000000001 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727270+0000", 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6571999999999998e-05 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727362+0000", 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2275000000000007e-05 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50727 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2351104~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697812+0000", 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1842450769999999, 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029725557999999999, 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.910 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50727 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697812+0000", 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.911 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697812+0000", 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697811+0000", 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697813+0000", 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.86e-06 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697819+0000", 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5169999999999999e-06 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697823+0000", 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4179999999999999e-06 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727385+0000", 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029562233 2024-09-13T05:02:45.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727461+0000", 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5541000000000001e-05 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727537+0000", 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6309000000000004e-05 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.914 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50728 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2392064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.18422947, 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029881569, 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50728 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697827+0000", 2024-09-13T05:02:45.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697829+0000", 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.897e-06 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697835+0000", 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6069999999999998e-06 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697839+0000", 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4059999999999998e-06 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727559+0000", 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029720135000000002 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727634+0000", 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4840999999999997e-05 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727709+0000", 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5012999999999995e-05 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50729 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2449408~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697843+0000", 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1842141420000001, 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030040001, 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.920 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50729 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697843+0000", 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697843+0000", 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697842+0000", 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697844+0000", 2024-09-13T05:02:45.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9980000000000002e-06 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697850+0000", 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5849999999999999e-06 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697855+0000", 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6260000000000003e-06 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727730+0000", 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029875432 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727805+0000", 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5171000000000002e-05 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727883+0000", 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7526000000000003e-05 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.924 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50730 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2490368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1841986200000001, 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030203912999999999, 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50730 2024-09-13T05:02:45.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697858+0000", 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697860+0000", 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9010000000000001e-06 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697865+0000", 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5629999999999999e-06 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697869+0000", 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0160000000000002e-06 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727904+0000", 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030034729999999999 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.727984+0000", 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9649999999999998e-05 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728062+0000", 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8369999999999997e-05 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.929 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50731 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2547712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.184183762, 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030453465999999998, 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50731 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697873+0000", 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697875+0000", 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9120000000000001e-06 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697880+0000", 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5339999999999999e-06 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697885+0000", 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.335e-06 2024-09-13T05:02:45.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728084+0000", 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030199756000000001 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728156+0000", 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1732999999999999e-05 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728327+0000", 2024-09-13T05:02:45.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017047399999999999 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50732 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2605056~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1841646640000008, 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030638446, 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50732 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697892+0000", 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697895+0000", 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3510000000000002e-06 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697901+0000", 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0769999999999996e-06 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.697906+0000", 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000003e-06 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728349+0000", 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030443606000000002 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728445+0000", 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5058999999999999e-05 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728531+0000", 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6069999999999994e-05 2024-09-13T05:02:45.938 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50733 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2646016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.698770+0000", 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1832867660000002, 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029940022, 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.939 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50733 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698770+0000", 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698770+0000", 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.940 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698769+0000", 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698773+0000", 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1659999999999996e-06 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698784+0000", 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1027e-05 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698791+0000", 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.443e-06 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728553+0000", 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029762365999999998 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728636+0000", 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2792000000000003e-05 2024-09-13T05:02:45.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728710+0000", 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4023000000000006e-05 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50734 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2703360~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1832599360000007, 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030084214000000001, 2024-09-13T05:02:45.943 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50734 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698797+0000", 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698799+0000", 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:02:45.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698805+0000", 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1029999999999998e-06 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698809+0000", 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5399999999999997e-06 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728732+0000", 2024-09-13T05:02:45.946 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029922338 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728804+0000", 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2305999999999995e-05 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728881+0000", 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7008999999999994e-05 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.947 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50735 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2801664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1832434060000008, 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030249990000000001, 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50735 2024-09-13T05:02:45.948 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.949 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698813+0000", 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698815+0000", 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8199999999999999e-06 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698820+0000", 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4179999999999996e-06 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.698825+0000", 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6199999999999998e-06 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728903+0000", 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030077948 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.728984+0000", 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1088999999999996e-05 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.951 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729063+0000", 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9375000000000005e-05 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50738 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 2957312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700805+0000", 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1812519819999991, 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028972106000000001, 2024-09-13T05:02:45.952 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50738 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700805+0000", 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.953 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700805+0000", 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700804+0000", 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700807+0000", 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3319999999999999e-06 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.954 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700814+0000", 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8229999999999997e-06 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700819+0000", 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.259e-06 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729573+0000", 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028754431 2024-09-13T05:02:45.955 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729670+0000", 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6217000000000004e-05 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729777+0000", 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00010744899999999999 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.956 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50739 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3055616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1812333349999999, 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029202947999999999, 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50739 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.957 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700823+0000", 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.958 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700825+0000", 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1349999999999999e-06 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700832+0000", 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7769999999999997e-06 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700838+0000", 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4419999999999997e-06 2024-09-13T05:02:45.959 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729822+0000", 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.028984566999999999 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.729911+0000", 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8702999999999994e-05 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730026+0000", 2024-09-13T05:02:45.960 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000115648 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50740 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3112960~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1812147230000001, 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029383038, 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.961 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50740 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.962 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700842+0000", 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700844+0000", 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1710000000000001e-06 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700850+0000", 2024-09-13T05:02:45.963 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5740000000000001e-06 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700856+0000", 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2320000000000001e-06 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730055+0000", 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029199583000000001 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.964 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730137+0000", 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1476000000000004e-05 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730225+0000", 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8363999999999996e-05 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50741 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.965 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1811965489999992, 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029627147999999999, 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50741 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:45.966 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700860+0000", 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.967 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700862+0000", 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700869+0000", 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5309999999999998e-06 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700874+0000", 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1499999999999998e-06 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.968 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730254+0000", 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029380074999999999 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730335+0000", 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1636000000000002e-05 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730487+0000", 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00015201599999999999 2024-09-13T05:02:45.969 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50742 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3211264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1811786689999995, 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029825695999999999, 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.970 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50742 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.971 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700878+0000", 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700880+0000", 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9920000000000002e-06 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700886+0000", 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3099999999999997e-06 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.972 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700891+0000", 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9130000000000002e-06 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730527+0000", 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029636445000000001 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730615+0000", 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7119000000000004e-05 2024-09-13T05:02:45.973 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730704+0000", 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9303999999999997e-05 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:50743 3.10 3:0f7a073d:::rbd_data.117c6173c20e.0000000000000000:14 [read 3268608~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.1811614299999995, 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.030066071999999999, 2024-09-13T05:02:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 50743 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.975 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700895+0000", 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700897+0000", 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.976 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700903+0000", 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.156e-06 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.700908+0000", 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.887e-06 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730767+0000", 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.029859041999999999 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.977 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730856+0000", 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8986000000000005e-05 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:37.730961+0000", 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000105252 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:45.978 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:45.979 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:45.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:45.982 INFO:teuthology.orchestra.run.smithi053.stdout:3704 write 0x71b53c4 thru 0x71c52ae (0xfeeb bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3706 writesame 0x31cb042 thru 0x31d7074 data_size 0x63(0xc033 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3707 punch from 0xd0cf7a to 0xd0e901, (0x1987 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3708 trunc from 0xaf38f4c to 0x47bb009 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3709 write 0xca5a04a thru 0xca5d8ae (0x3865 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3710 read 0xba23a2b thru 0xba2f7c9 (0xbd9f bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3711 punch from 0xacfc949 to 0xad04e2b, (0x84e2 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3712 writesame 0x42ad032 thru 0x42b1dc7 data_size 0x1(0x4d96 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3714 read 0x94fb4a3 thru 0x9508b42 (0xd6a0 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3716 read 0x930c164 thru 0x930d617 (0x14b4 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3717 punch from 0x92114ee to 0x9215c18, (0x472a bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3718 read 0x9089321 thru 0x908eb2e (0x580e bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3719 writesame 0x59a13ac thru 0x59a9a4e data_size 0x1(0x86a3 bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3720 punch from 0x50e6806 to 0x50f5203, (0xe9fd bytes) 2024-09-13T05:02:45.983 INFO:teuthology.orchestra.run.smithi053.stdout:3721 write 0x5338abb thru 0x533c568 (0x3aae bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3722 read 0x5064497 thru 0x506e5fc (0xa166 bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3723 punch from 0xa0870c to 0xa16d9e, (0xe692 bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3724 read 0xb281403 thru 0xb281bef (0x7ed bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3725 write 0xb215bf3 thru 0xb223767 (0xdb75 bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3726 write 0xa3a0989 thru 0xa3ad878 (0xcef0 bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3727 punch from 0x56952d2 to 0x56a24cf, (0xd1fd bytes) 2024-09-13T05:02:45.984 INFO:teuthology.orchestra.run.smithi053.stdout:3728 writesame 0x8f5de1a thru 0x8f6900b data_size 0x1(0xb1f2 bytes) 2024-09-13T05:02:45.984 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:45.978+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:46.001 INFO:teuthology.orchestra.run.smithi053.stdout:3729 write 0xe000e8a thru 0xe009273 (0x83ea bytes) 2024-09-13T05:02:46.004 INFO:teuthology.orchestra.run.smithi053.stdout:3730 read 0xc61ebe3 thru 0xc62b760 (0xcb7e bytes) 2024-09-13T05:02:46.004 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.010 INFO:teuthology.orchestra.run.smithi053.stdout:3734 trunc from 0xe009274 to 0x8c30800 2024-09-13T05:02:46.025 INFO:teuthology.orchestra.run.smithi053.stdout:3735 write 0x2e9f4db thru 0x2ea3d22 (0x4848 bytes) 2024-09-13T05:02:46.028 INFO:teuthology.orchestra.run.smithi053.stdout:3736 punch from 0x569059 to 0x57649c, (0xd443 bytes) 2024-09-13T05:02:46.028 INFO:teuthology.orchestra.run.smithi053.stdout:3737 write 0x101a0aa thru 0x101e564 (0x44bb bytes) 2024-09-13T05:02:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:3738 read 0x165d785 thru 0x166d755 (0xffd1 bytes) 2024-09-13T05:02:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:3739 read 0x3d2eae3 thru 0x3d3638a (0x78a8 bytes) 2024-09-13T05:02:46.030 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.042 INFO:teuthology.orchestra.run.smithi053.stdout:3741 write 0xd99837a thru 0xd99e2d3 (0x5f5a bytes) 2024-09-13T05:02:46.045 INFO:teuthology.orchestra.run.smithi053.stdout:3742 read 0x51ca085 thru 0x51d2d7d (0x8cf9 bytes) 2024-09-13T05:02:46.045 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.045 INFO:teuthology.orchestra.run.smithi053.stdout:3744 read 0x597a6d4 thru 0x598781c (0xd149 bytes) 2024-09-13T05:02:46.053 INFO:teuthology.orchestra.run.smithi053.stdout:3745 trunc from 0xd99e2d4 to 0x5c4895f 2024-09-13T05:02:46.065 INFO:teuthology.orchestra.run.smithi053.stdout:3746 write 0x1d58da2 thru 0x1d5cce1 (0x3f40 bytes) 2024-09-13T05:02:46.066 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.066 INFO:teuthology.orchestra.run.smithi053.stdout:3748 read 0x12668fd thru 0x1276108 (0xf80c bytes) 2024-09-13T05:02:46.066 INFO:teuthology.orchestra.run.smithi053.stdout:3749 read 0x297bf7e thru 0x298447c (0x84ff bytes) 2024-09-13T05:02:46.066 INFO:teuthology.orchestra.run.smithi053.stdout:3750 read 0xa74ed5 thru 0xa75309 (0x435 bytes) 2024-09-13T05:02:46.066 INFO:teuthology.orchestra.run.smithi053.stdout:3751 writesame 0x488d735 thru 0x489afdf data_size 0x3(0xd8ab bytes) 2024-09-13T05:02:46.080 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:46.080+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:46.094 INFO:teuthology.orchestra.run.smithi053.stdout:3753 write 0xeb4b7da thru 0xeb5565a (0x9e81 bytes) 2024-09-13T05:02:46.094 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:46.094 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:46.094 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:46.094 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:46.097 INFO:teuthology.orchestra.run.smithi053.stdout:3754 write 0xc036e52 thru 0xc038276 (0x1425 bytes) 2024-09-13T05:02:46.100 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.101 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.104 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:46.115 INFO:teuthology.orchestra.run.smithi053.stdout:3760 trunc from 0xeb5565b to 0x1677274 2024-09-13T05:02:46.129 INFO:teuthology.orchestra.run.smithi053.stdout:3762 punch from 0xb330a5 to 0xb34f38, (0x1e93 bytes) 2024-09-13T05:02:46.129 INFO:teuthology.orchestra.run.smithi053.stdout:3763 punch from 0x24094 to 0x291d3, (0x513f bytes) 2024-09-13T05:02:46.129 INFO:teuthology.orchestra.run.smithi053.stdout:3764 punch from 0x34d836 to 0x35bc07, (0xe3d1 bytes) 2024-09-13T05:02:46.129 INFO:teuthology.orchestra.run.smithi053.stdout:3765 punch from 0x10866fd to 0x1095f8c, (0xf88f bytes) 2024-09-13T05:02:46.134 INFO:teuthology.orchestra.run.smithi053.stdout:3766 writesame 0x20daf89 thru 0x20dd85c data_size 0x1a(0x28d4 bytes) 2024-09-13T05:02:46.135 INFO:teuthology.orchestra.run.smithi053.stdout:3767 read 0x1960b51 thru 0x1964052 (0x3502 bytes) 2024-09-13T05:02:46.146 INFO:teuthology.orchestra.run.smithi053.stdout:3768 trunc from 0x20dd85d to 0xcb74b20 2024-09-13T05:02:46.152 INFO:teuthology.orchestra.run.smithi053.stdout:3769 write 0x64df1bc thru 0x64e5288 (0x60cd bytes) 2024-09-13T05:02:46.156 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.157 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.159 INFO:teuthology.orchestra.run.smithi053.stdout:3774 trunc from 0xcb74b20 to 0xa9c5b7a 2024-09-13T05:02:46.164 INFO:teuthology.orchestra.run.smithi053.stdout:3775 write 0x7ddeada thru 0x7de3b27 (0x504e bytes) 2024-09-13T05:02:46.176 INFO:teuthology.orchestra.run.smithi053.stdout:3777 trunc from 0xa9c5b7a to 0x3614a14 2024-09-13T05:02:46.181 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:46.180+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:46.187 INFO:teuthology.orchestra.run.smithi053.stdout:3778 punch from 0x2b2188c to 0x2b28ff5, (0x7769 bytes) 2024-09-13T05:02:46.187 INFO:teuthology.orchestra.run.smithi053.stdout:3779 punch from 0x2e92dab to 0x2e9b38b, (0x85e0 bytes) 2024-09-13T05:02:46.187 INFO:teuthology.orchestra.run.smithi053.stdout:3780 read 0x2393192 thru 0x239bfd5 (0x8e44 bytes) 2024-09-13T05:02:46.187 INFO:teuthology.orchestra.run.smithi053.stdout:3781 read 0x245e25a thru 0x246d52a (0xf2d1 bytes) 2024-09-13T05:02:46.193 INFO:teuthology.orchestra.run.smithi053.stdout:3782 write 0x369de6a thru 0x36a57ae (0x7945 bytes) 2024-09-13T05:02:46.216 INFO:teuthology.orchestra.run.smithi053.stdout:3783 writesame 0xe8b6213 thru 0xe8b8491 data_size 0x1(0x227f bytes) 2024-09-13T05:02:46.216 INFO:teuthology.orchestra.run.smithi053.stdout:3784 writesame 0x75b577a thru 0x75b8607 data_size 0x2(0x2e8e bytes) 2024-09-13T05:02:46.221 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.221 INFO:teuthology.orchestra.run.smithi053.stdout:3786 trunc from 0xe8b8492 to 0xea2f7fb 2024-09-13T05:02:46.227 INFO:teuthology.orchestra.run.smithi053.stdout:3788 read 0x6f23cce thru 0x6f2ac8b (0x6fbe bytes) 2024-09-13T05:02:46.227 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.227 INFO:teuthology.orchestra.run.smithi053.stdout:3793 read 0x295aa23 thru 0x2966a9e (0xc07c bytes) 2024-09-13T05:02:46.228 INFO:teuthology.orchestra.run.smithi053.stdout:3794 write 0x2d4083d thru 0x2d46117 (0x58db bytes) 2024-09-13T05:02:46.230 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.230 INFO:teuthology.orchestra.run.smithi053.stdout:3802 punch from 0xc6bd0fb to 0xc6bf330, (0x2235 bytes) 2024-09-13T05:02:46.244 INFO:teuthology.orchestra.run.smithi053.stdout:3805 trunc from 0xea2f7fb to 0x2c2e8ea 2024-09-13T05:02:46.253 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:46.253 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:46.254 INFO:teuthology.orchestra.run.smithi053.stdout:3806 read 0x29ba0f thru 0x2a58fe (0x9ef0 bytes) 2024-09-13T05:02:46.255 INFO:teuthology.orchestra.run.smithi053.stdout:3807 read 0x25a756d thru 0x25adf28 (0x69bc bytes) 2024-09-13T05:02:46.262 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:46.267 INFO:teuthology.orchestra.run.smithi053.stdout:3808 write 0x7108b73 thru 0x710e4f8 (0x5986 bytes) 2024-09-13T05:02:46.271 INFO:teuthology.orchestra.run.smithi053.stdout:3810 read 0x6ffd441 thru 0x6ffdcb6 (0x876 bytes) 2024-09-13T05:02:46.271 INFO:teuthology.orchestra.run.smithi053.stdout:3811 punch from 0xbf16d8 to 0xbf2374, (0xc9c bytes) 2024-09-13T05:02:46.271 INFO:teuthology.orchestra.run.smithi053.stdout:3812 punch from 0x613cc75 to 0x614bf9a, (0xf325 bytes) 2024-09-13T05:02:46.271 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.271 INFO:teuthology.orchestra.run.smithi053.stdout:3814 read 0x4d9fc21 thru 0x4daaaff (0xaedf bytes) 2024-09-13T05:02:46.271 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.275 INFO:teuthology.orchestra.run.smithi053.stdout:3816 trunc from 0x710e4f9 to 0x2da0c33 2024-09-13T05:02:46.282 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:46.281+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:46.286 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.289 INFO:teuthology.orchestra.run.smithi053.stdout:3818 trunc from 0x2da0c33 to 0x8eda76 2024-09-13T05:02:46.324 INFO:teuthology.orchestra.run.smithi053.stdout:3819 write 0x2232bcc thru 0x2238fee (0x6423 bytes) 2024-09-13T05:02:46.331 INFO:teuthology.orchestra.run.smithi053.stdout:3820 trunc from 0x2238fef to 0x5917893 2024-09-13T05:02:46.347 INFO:teuthology.orchestra.run.smithi053.stdout:3821 writesame 0x85c7880 thru 0x85d0759 data_size 0x45(0x8eda bytes) 2024-09-13T05:02:46.347 INFO:teuthology.orchestra.run.smithi053.stdout:3825 punch from 0x40063e7 to 0x400ac24, (0x483d bytes) 2024-09-13T05:02:46.347 INFO:teuthology.orchestra.run.smithi053.stdout:3827 writesame 0x848d02e thru 0x848fadd data_size 0x8(0x2ab0 bytes) 2024-09-13T05:02:46.357 INFO:teuthology.orchestra.run.smithi053.stdout:3828 write 0x9900cbf thru 0x9904a3b (0x3d7d bytes) 2024-09-13T05:02:46.361 INFO:teuthology.orchestra.run.smithi053.stdout:3830 read 0x8db888d thru 0x8dbda49 (0x51bd bytes) 2024-09-13T05:02:46.361 INFO:teuthology.orchestra.run.smithi053.stdout:3832 write 0x331e670 thru 0x3321356 (0x2ce7 bytes) 2024-09-13T05:02:46.365 INFO:teuthology.orchestra.run.smithi053.stdout:3833 read 0x127763 thru 0x12e179 (0x6a17 bytes) 2024-09-13T05:02:46.368 INFO:teuthology.orchestra.run.smithi053.stdout:3834 trunc from 0x9904a3c to 0x7133c82 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:46.378 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.147624155999999, 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.381 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.146696046999999, 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.383 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:02:46.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:02:46.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.388 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.145597638000002, 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:02:46.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:02:46.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.393 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.139537677, 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.394 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:02:46.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:02:46.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.139528682000002, 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:02:46.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:02:46.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.404 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.139520310999998, 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:02:46.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.13951029, 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:02:46.409 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:02:46.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.413 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.139494567, 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:02:46.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.415 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:02:46.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.137592397999999, 2024-09-13T05:02:46.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:02:46.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:02:46.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.137584439000001, 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.423 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.137576751000001, 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.428 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:02:46.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.137566059000001, 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.432 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:46.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:02:46.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.137557818000001, 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.437 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:02:46.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.441 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.137534157000001, 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:46.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:02:46.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.135891698000002, 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.450 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.135883464999999, 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.451 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:02:46.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.135660735999998, 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.455 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:46.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.135648187000001, 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:46.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.133608961, 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.464 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:02:46.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:02:46.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:46.469 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:46.470 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 22.133599182000001, 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:02:46.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:46.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:46.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:46.476 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:46.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3835 writesame 0x630ad88 thru 0x630d7d2 data_size 0x9(0x2a4b bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3837 write 0x28fd468 thru 0x2901505 (0x409e bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3838 read 0x606c6f8 thru 0x6077c0c (0xb515 bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3840 write 0xd677007 thru 0xd680b43 (0x9b3d bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3841 read 0x1489e8 thru 0x1571e9 (0xe802 bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3843 writesame 0xc0ab85 thru 0xc16ba5 data_size 0xf(0xc021 bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3844 write 0x5fecc27 thru 0x5ffcc03 (0xffdd bytes) 2024-09-13T05:02:46.480 INFO:teuthology.orchestra.run.smithi053.stdout:3845 trunc from 0xd680b44 to 0xc68dde2 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3851 read 0x534ed15 thru 0x5353892 (0x4b7e bytes) 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3853 punch from 0x8a8a5d1 to 0x8a948d1, (0xa300 bytes) 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3855 trunc from 0xc68dde2 to 0x495766b 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3858 trunc from 0x495766b to 0x9475318 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3860 trunc from 0x9475318 to 0xcf89572 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3863 writesame 0x17df944 thru 0x17ec9ae data_size 0x5(0xd06b bytes) 2024-09-13T05:02:46.481 INFO:teuthology.orchestra.run.smithi053.stdout:3864 write 0xecaa4a8 thru 0xecad30e (0x2e67 bytes) 2024-09-13T05:02:46.481 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:46.476+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:46.482 INFO:teuthology.orchestra.run.smithi053.stdout:3865 punch from 0xdd6b29c to 0xdd798bf, (0xe623 bytes) 2024-09-13T05:02:46.482 INFO:teuthology.orchestra.run.smithi053.stdout:3866 punch from 0x6c6b364 to 0x6c752ab, (0x9f47 bytes) 2024-09-13T05:02:46.482 INFO:teuthology.orchestra.run.smithi053.stdout:3867 read 0x9c92fc7 thru 0x9c938f9 (0x933 bytes) 2024-09-13T05:02:46.482 INFO:teuthology.orchestra.run.smithi053.stdout:3868 read 0x2e4f639 thru 0x2e5f0b9 (0xfa81 bytes) 2024-09-13T05:02:46.482 INFO:teuthology.orchestra.run.smithi053.stdout:3869 punch from 0x4e07d5f to 0x4e091bd, (0x145e bytes) 2024-09-13T05:02:46.483 INFO:teuthology.orchestra.run.smithi053.stdout:3871 write 0xc74afbb thru 0xc753274 (0x82ba bytes) 2024-09-13T05:02:46.484 INFO:teuthology.orchestra.run.smithi053.stdout:3872 punch from 0x6943871 to 0x694d627, (0x9db6 bytes) 2024-09-13T05:02:46.485 INFO:teuthology.orchestra.run.smithi053.stdout:3873 write 0x18dd2d4 thru 0x18eaf15 (0xdc42 bytes) 2024-09-13T05:02:46.488 INFO:teuthology.orchestra.run.smithi053.stdout:3875 writesame 0xd7ca5d7 thru 0xd7d8080 data_size 0xd(0xdaaa bytes) 2024-09-13T05:02:46.494 INFO:teuthology.orchestra.run.smithi053.stdout:3877 write 0xbddf406 thru 0xbde572a (0x6325 bytes) 2024-09-13T05:02:46.498 INFO:teuthology.orchestra.run.smithi053.stdout:3881 writesame 0xe58357d thru 0xe58b71e data_size 0x3(0x81a2 bytes) 2024-09-13T05:02:46.509 INFO:teuthology.orchestra.run.smithi053.stdout:3882 trunc from 0xecad30f to 0xabe5d67 2024-09-13T05:02:46.521 INFO:teuthology.orchestra.run.smithi053.stdout:3883 read 0x36febba thru 0x370593c (0x6d83 bytes) 2024-09-13T05:02:46.528 INFO:teuthology.orchestra.run.smithi053.stdout:3886 writesame 0xc0f9d7a thru 0xc101e36 data_size 0x1(0x80bd bytes) 2024-09-13T05:02:46.528 INFO:teuthology.orchestra.run.smithi053.stdout:3887 write 0x29217d9 thru 0x2922edc (0x1704 bytes) 2024-09-13T05:02:46.531 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.531 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.531 INFO:teuthology.orchestra.run.smithi053.stdout:3893 punch from 0x794240a to 0x794ae42, (0x8a38 bytes) 2024-09-13T05:02:46.531 INFO:teuthology.orchestra.run.smithi053.stdout:3894 write 0x1de4e20 thru 0x1defc44 (0xae25 bytes) 2024-09-13T05:02:46.534 INFO:teuthology.orchestra.run.smithi053.stdout:3895 read 0x8883ae7 thru 0x8888a05 (0x4f1f bytes) 2024-09-13T05:02:46.547 INFO:teuthology.orchestra.run.smithi053.stdout:3896 trunc from 0xc101e37 to 0xae8cc2 2024-09-13T05:02:46.572 INFO:teuthology.orchestra.run.smithi053.stdout:3898 writesame 0x6ac12f9 thru 0x6acc704 data_size 0x8a(0xb40c bytes) 2024-09-13T05:02:46.578 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:46.576+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:46.579 INFO:teuthology.orchestra.run.smithi053.stdout:3899 write 0x74649ce thru 0x746879f (0x3dd2 bytes) 2024-09-13T05:02:46.585 INFO:teuthology.orchestra.run.smithi053.stdout:3900 trunc from 0x74687a0 to 0x9bd2104 2024-09-13T05:02:46.590 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.590 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.590 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.591 INFO:teuthology.orchestra.run.smithi053.stdout:3905 read 0x2a194c7 thru 0x2a2939d (0xfed7 bytes) 2024-09-13T05:02:46.591 INFO:teuthology.orchestra.run.smithi053.stdout:3906 punch from 0x3e0a917 to 0x3e0d9fe, (0x30e7 bytes) 2024-09-13T05:02:46.591 INFO:teuthology.orchestra.run.smithi053.stdout:3907 read 0x62657fb thru 0x6272527 (0xcd2d bytes) 2024-09-13T05:02:46.591 INFO:teuthology.orchestra.run.smithi053.stdout:3909 trunc from 0x9bd2104 to 0x944415c 2024-09-13T05:02:46.596 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.596 INFO:teuthology.orchestra.run.smithi053.stdout:3914 punch from 0x8c0bbc9 to 0x8c14e8d, (0x92c4 bytes) 2024-09-13T05:02:46.603 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:46.603 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:46.603 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:46.603 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:46.605 INFO:teuthology.orchestra.run.smithi053.stdout:3915 writesame 0xb43fbf9 thru 0xb4416ae data_size 0xd(0x1ab6 bytes) 2024-09-13T05:02:46.606 INFO:teuthology.orchestra.run.smithi053.stdout:3917 read 0x7ce1def thru 0x7cef65a (0xd86c bytes) 2024-09-13T05:02:46.606 INFO:teuthology.orchestra.run.smithi053.stdout:3918 punch from 0x40409f2 to 0x40466d2, (0x5ce0 bytes) 2024-09-13T05:02:46.606 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.606 INFO:teuthology.orchestra.run.smithi053.stdout:3923 writesame 0x83dacc7 thru 0x83e6e12 data_size 0x4(0xc14c bytes) 2024-09-13T05:02:46.613 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:46.613 INFO:teuthology.orchestra.run.smithi053.stdout:3924 trunc from 0xb4416af to 0xb015709 2024-09-13T05:02:46.619 INFO:teuthology.orchestra.run.smithi053.stdout:3925 writesame 0x8a0ed83 thru 0x8a18fca data_size 0x24(0xa248 bytes) 2024-09-13T05:02:46.624 INFO:teuthology.orchestra.run.smithi053.stdout:3926 read 0x534b76f thru 0x5356887 (0xb119 bytes) 2024-09-13T05:02:46.624 INFO:teuthology.orchestra.run.smithi053.stdout:3927 punch from 0x403d1b to 0x40b794, (0x7a79 bytes) 2024-09-13T05:02:46.625 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.625 INFO:teuthology.orchestra.run.smithi053.stdout:3930 writesame 0x1a0beb2 thru 0x1a1b6ef data_size 0x9b(0xf83e bytes) 2024-09-13T05:02:46.629 INFO:teuthology.orchestra.run.smithi053.stdout:3931 write 0x12d882b thru 0x12e100e (0x87e4 bytes) 2024-09-13T05:02:46.633 INFO:teuthology.orchestra.run.smithi053.stdout:3932 punch from 0x1af08a2 to 0x1af5c6e, (0x53cc bytes) 2024-09-13T05:02:46.633 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.633 INFO:teuthology.orchestra.run.smithi053.stdout:3936 read 0x1dd319e thru 0x1ddcf5c (0x9dbf bytes) 2024-09-13T05:02:46.633 INFO:teuthology.orchestra.run.smithi053.stdout:3938 punch from 0x6d425ae to 0x6d50cdb, (0xe72d bytes) 2024-09-13T05:02:46.633 INFO:teuthology.orchestra.run.smithi053.stdout:3939 read 0x786b516 thru 0x787884b (0xd336 bytes) 2024-09-13T05:02:46.637 INFO:teuthology.orchestra.run.smithi053.stdout:3942 trunc from 0xb015709 to 0x747226c 2024-09-13T05:02:46.652 INFO:teuthology.orchestra.run.smithi053.stdout:3943 trunc from 0x747226c to 0x4a681d7 2024-09-13T05:02:46.670 INFO:teuthology.orchestra.run.smithi053.stdout:3944 writesame 0x5f7c2a5 thru 0x5f82ed9 data_size 0x1(0x6c35 bytes) 2024-09-13T05:02:46.670 INFO:teuthology.orchestra.run.smithi053.stdout:3945 write 0x20e2e1c thru 0x20eba68 (0x8c4d bytes) 2024-09-13T05:02:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:3948 read 0x48a0134 thru 0x48a7b90 (0x7a5d bytes) 2024-09-13T05:02:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:3949 read 0xa00e12 thru 0xa0f207 (0xe3f6 bytes) 2024-09-13T05:02:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:3950 read 0x4e35b2e thru 0x4e40c87 (0xb15a bytes) 2024-09-13T05:02:46.674 INFO:teuthology.orchestra.run.smithi053.stdout:3951 read 0x2ab0fe1 thru 0x2ab944e (0x846e bytes) 2024-09-13T05:02:46.676 INFO:teuthology.orchestra.run.smithi053.stdout:3952 trunc from 0x5f82eda to 0x8094897 2024-09-13T05:02:46.679 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:46.678+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:46.682 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.683 INFO:teuthology.orchestra.run.smithi053.stdout:3955 trunc from 0x8094897 to 0x8964701 2024-09-13T05:02:46.688 INFO:teuthology.orchestra.run.smithi053.stdout:3958 write 0x74076c5 thru 0x740ca55 (0x5391 bytes) 2024-09-13T05:02:46.692 INFO:teuthology.orchestra.run.smithi053.stdout:3959 write 0x3feaa8e thru 0x3ff5e5c (0xb3cf bytes) 2024-09-13T05:02:46.707 INFO:teuthology.orchestra.run.smithi053.stdout:3961 writesame 0xdb32c66 thru 0xdb3c8c7 data_size 0x25(0x9c62 bytes) 2024-09-13T05:02:46.707 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.707 INFO:teuthology.orchestra.run.smithi053.stdout:3965 write 0x3b9a700 thru 0x3ba91e2 (0xeae3 bytes) 2024-09-13T05:02:46.712 INFO:teuthology.orchestra.run.smithi053.stdout:3967 punch from 0x18eb20c to 0x18f92aa, (0xe09e bytes) 2024-09-13T05:02:46.721 INFO:teuthology.orchestra.run.smithi053.stdout:3969 trunc from 0xdb3c8c8 to 0x5aed1df 2024-09-13T05:02:46.730 INFO:teuthology.orchestra.run.smithi053.stdout:3970 read 0x37d6eb8 thru 0x37e02af (0x93f8 bytes) 2024-09-13T05:02:46.732 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:46.732 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:46.732 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:46.733 INFO:teuthology.orchestra.run.smithi053.stdout:3972 trunc from 0x5aed1df to 0x3689589 2024-09-13T05:02:46.742 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:46.750 INFO:teuthology.orchestra.run.smithi053.stdout:3975 trunc from 0x3689589 to 0x8357588 2024-09-13T05:02:46.755 INFO:teuthology.orchestra.run.smithi053.stdout:3978 write 0xd65239 thru 0xd732e5 (0xe0ad bytes) 2024-09-13T05:02:46.759 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.760 INFO:teuthology.orchestra.run.smithi053.stdout:3981 trunc from 0x8357588 to 0x7b3bebe 2024-09-13T05:02:46.765 INFO:teuthology.orchestra.run.smithi053.stdout:3982 read 0x3be47c5 thru 0x3bebf42 (0x777e bytes) 2024-09-13T05:02:46.778 INFO:teuthology.orchestra.run.smithi053.stdout:3983 write 0xd38edbc thru 0xd39c55a (0xd79f bytes) 2024-09-13T05:02:46.779 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:46.777+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:46.782 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.782 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.782 INFO:teuthology.orchestra.run.smithi053.stdout:3987 read 0x7e99c4f thru 0x7e9c8f4 (0x2ca6 bytes) 2024-09-13T05:02:46.782 INFO:teuthology.orchestra.run.smithi053.stdout:3988 read 0x5f9970e thru 0x5fa7d65 (0xe658 bytes) 2024-09-13T05:02:46.790 INFO:teuthology.orchestra.run.smithi053.stdout:3991 trunc from 0xd39c55b to 0x6a095b6 2024-09-13T05:02:46.801 INFO:teuthology.orchestra.run.smithi053.stdout:3993 read 0x20dd87b thru 0x20ea240 (0xc9c6 bytes) 2024-09-13T05:02:46.801 INFO:teuthology.orchestra.run.smithi053.stdout:3995 punch from 0x3737f0d to 0x373ea1a, (0x6b0d bytes) 2024-09-13T05:02:46.802 INFO:teuthology.orchestra.run.smithi053.stdout:3996 read 0x8e2a0e thru 0x8ebe59 (0x944c bytes) 2024-09-13T05:02:46.802 INFO:teuthology.orchestra.run.smithi053.stdout:3997 read 0x2fa5d1c thru 0x2fa8fdd (0x32c2 bytes) 2024-09-13T05:02:46.802 INFO:teuthology.orchestra.run.smithi053.stdout:3998 write 0x16184cf thru 0x1621ea8 (0x99da bytes) 2024-09-13T05:02:46.820 INFO:teuthology.orchestra.run.smithi053.stdout:4000 write 0xd40ce96 thru 0xd412ebb (0x6026 bytes) 2024-09-13T05:02:46.833 INFO:teuthology.orchestra.run.smithi053.stdout:4001 writesame 0xec631c4 thru 0xec71cae data_size 0x1(0xeaeb bytes) 2024-09-13T05:02:46.833 INFO:teuthology.orchestra.run.smithi053.stdout:4002 read 0x759cf0a thru 0x75a9ccb (0xcdc2 bytes) 2024-09-13T05:02:46.833 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.833 INFO:teuthology.orchestra.run.smithi053.stdout:4007 read 0x4dda1d6 thru 0x4ddf843 (0x566e bytes) 2024-09-13T05:02:46.833 INFO:teuthology.orchestra.run.smithi053.stdout:4008 write 0xd336c60 thru 0xd338797 (0x1b38 bytes) 2024-09-13T05:02:46.837 INFO:teuthology.orchestra.run.smithi053.stdout:4009 read 0xdba80f1 thru 0xdbabfbf (0x3ecf bytes) 2024-09-13T05:02:46.837 INFO:teuthology.orchestra.run.smithi053.stdout:4010 write 0x818e79d thru 0x819c0ab (0xd90f bytes) 2024-09-13T05:02:46.842 INFO:teuthology.orchestra.run.smithi053.stdout:4012 writesame 0x2b1102c thru 0x2b155a1 data_size 0x11(0x4576 bytes) 2024-09-13T05:02:46.860 INFO:teuthology.orchestra.run.smithi053.stdout:4013 trunc from 0xec71caf to 0x1915875 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215351448, 2024-09-13T05:02:46.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215335451, 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.880 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215318655000001, 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.881 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215301682, 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:02:46.885 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.887 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:02:46.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.889 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215285288, 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.890 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:02:46.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:46.895 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215268252, 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:02:46.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:02:46.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215234669000001, 2024-09-13T05:02:46.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:02:46.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:02:46.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215215540000001, 2024-09-13T05:02:46.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:02:46.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:46.907 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215137671000001, 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.911 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215076926, 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:46.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.915 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.215016712000001, 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:02:46.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214999385, 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:02:46.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214970213000001, 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.924 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:46.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214950528999999, 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.928 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214933095999999, 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:02:46.932 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:02:46.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:46.936 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214915454, 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:02:46.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:46.940 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214881278, 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214855928, 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.945 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:46.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:02:46.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214838255, 2024-09-13T05:02:46.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.953 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.214814050999999, 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:46.954 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:46.957 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:46.959 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:46.958+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:46.959 INFO:teuthology.orchestra.run.smithi053.stdout:4015 read 0x14e67ea thru 0x14f15cd (0xade4 bytes) 2024-09-13T05:02:46.959 INFO:teuthology.orchestra.run.smithi053.stdout:4016 read 0x95647c thru 0x964d79 (0xe8fe bytes) 2024-09-13T05:02:46.959 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.959 INFO:teuthology.orchestra.run.smithi053.stdout:4020 read 0x798703 thru 0x79b992 (0x3290 bytes) 2024-09-13T05:02:46.959 INFO:teuthology.orchestra.run.smithi053.stdout:4021 trunc from 0x1915875 to 0x3ccd059 2024-09-13T05:02:46.960 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4022 read 0x26cda1 thru 0x26df48 (0x11a8 bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4023 read 0x2fe39aa thru 0x2ff2040 (0xe697 bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4024 write 0xc1d4511 thru 0xc1e3852 (0xf342 bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4025 trunc from 0xc1e3853 to 0xb4e11f4 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4026 read 0xa65919f thru 0xa66844a (0xf2ac bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4027 punch from 0x6c74aa3 to 0x6c81d60, (0xd2bd bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4028 read 0x355472a thru 0x3559076 (0x494d bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4029 punch from 0x34c6b93 to 0x34d477f, (0xdbec bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4031 write 0x14b572e thru 0x14be8fe (0x91d1 bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4034 write 0x706c55 thru 0x70d000 (0x63ac bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4035 write 0x8b7502c thru 0x8b7a795 (0x576a bytes) 2024-09-13T05:02:46.963 INFO:teuthology.orchestra.run.smithi053.stdout:4038 read 0x176d4d3 thru 0x1775e51 (0x897f bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4039 punch from 0x4f322de to 0x4f3f2f4, (0xd016 bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4040 punch from 0x4926192 to 0x4927e1b, (0x1c89 bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4042 write 0x2235be8 thru 0x223f801 (0x9c1a bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4044 writesame 0x1eda0c3 thru 0x1edcd9b data_size 0x3(0x2cd9 bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4045 write 0x6466efc thru 0x646da97 (0x6b9c bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4046 writesame 0x6a17531 thru 0x6a192ad data_size 0x1(0x1d7d bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4048 writesame 0x1f76a3 thru 0x1fec3d data_size 0x4d(0x759b bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4049 punch from 0x476ecaf to 0x47749f0, (0x5d41 bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4050 write 0x88994b4 thru 0x889f6ac (0x61f9 bytes) 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:4051 trunc from 0xb4e11f4 to 0xb597bf9 2024-09-13T05:02:46.964 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.965 INFO:teuthology.orchestra.run.smithi053.stdout:4053 write 0xd2e83b2 thru 0xd2ef6c9 (0x7318 bytes) 2024-09-13T05:02:46.967 INFO:teuthology.orchestra.run.smithi053.stdout:4054 read 0x3a5d21e thru 0x3a69a30 (0xc813 bytes) 2024-09-13T05:02:46.967 INFO:teuthology.orchestra.run.smithi053.stdout:4055 write 0x2c018e5 thru 0x2c0859c (0x6cb8 bytes) 2024-09-13T05:02:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:4056 punch from 0x4f3b213 to 0x4f44bd8, (0x99c5 bytes) 2024-09-13T05:02:46.970 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:4058 writesame 0xe61b2f0 thru 0xe62362e data_size 0x1(0x833f bytes) 2024-09-13T05:02:46.976 INFO:teuthology.orchestra.run.smithi053.stdout:4059 write 0x5ed87f9 thru 0x5edd5c2 (0x4dca bytes) 2024-09-13T05:02:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:4060 punch from 0x7b53757 to 0x7b58c26, (0x54cf bytes) 2024-09-13T05:02:46.980 INFO:teuthology.orchestra.run.smithi053.stdout:4061 writesame 0x2785fa3 thru 0x2794582 data_size 0x30(0xe5e0 bytes) 2024-09-13T05:02:46.984 INFO:teuthology.orchestra.run.smithi053.stdout:4062 read 0x2085dd1 thru 0x2092628 (0xc858 bytes) 2024-09-13T05:02:46.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:46.998 INFO:teuthology.orchestra.run.smithi053.stdout:4063 trunc from 0xe62362f to 0x1eb009c 2024-09-13T05:02:47.013 INFO:teuthology.orchestra.run.smithi053.stdout:4064 read 0x61ef80 thru 0x62529f (0x6320 bytes) 2024-09-13T05:02:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:4066 read 0x3ad498 thru 0x3ba8ee (0xd457 bytes) 2024-09-13T05:02:47.014 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:02:47.036 INFO:teuthology.orchestra.run.smithi053.stdout:4070 writesame 0xeda0c12 thru 0xeda949d data_size 0x12(0x888c bytes) 2024-09-13T05:02:47.046 INFO:teuthology.orchestra.run.smithi053.stdout:4071 trunc from 0xeda949e to 0x624e34c 2024-09-13T05:02:47.052 INFO:teuthology.orchestra.run.smithi053.stdout:4072 write 0x5e6b707 thru 0x5e6f84c (0x4146 bytes) 2024-09-13T05:02:47.056 INFO:teuthology.orchestra.run.smithi053.stdout:4073 read 0x53e63eb thru 0x53f4bca (0xe7e0 bytes) 2024-09-13T05:02:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:4074 read 0x7f909 thru 0x83740 (0x3e38 bytes) 2024-09-13T05:02:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.057 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.059 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:47.058+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:47.065 INFO:teuthology.orchestra.run.smithi053.stdout:4080 trunc from 0x624e34c to 0xd6bafdc 2024-09-13T05:02:47.079 INFO:teuthology.orchestra.run.smithi053.stdout:4081 trunc from 0xd6bafdc to 0x674d61b 2024-09-13T05:02:47.085 INFO:teuthology.orchestra.run.smithi053.stdout:4083 read 0x33f715f thru 0x33f9bc3 (0x2a65 bytes) 2024-09-13T05:02:47.087 INFO:teuthology.orchestra.run.smithi053.stdout:4085 trunc from 0x674d61b to 0x8e9a3ce 2024-09-13T05:02:47.093 INFO:teuthology.orchestra.run.smithi053.stdout:4086 punch from 0x754394f to 0x754d0d4, (0x9785 bytes) 2024-09-13T05:02:47.094 INFO:teuthology.orchestra.run.smithi053.stdout:4088 write 0x658ef57 thru 0x659d84f (0xe8f9 bytes) 2024-09-13T05:02:47.103 INFO:teuthology.orchestra.run.smithi053.stdout:4090 writesame 0x8f1b22e thru 0x8f1e2c3 data_size 0x9(0x3096 bytes) 2024-09-13T05:02:47.104 INFO:teuthology.orchestra.run.smithi053.stdout:4093 read 0x46de03a thru 0x46e232e (0x42f5 bytes) 2024-09-13T05:02:47.107 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:47.107 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:47.107 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:47.112 INFO:teuthology.orchestra.run.smithi053.stdout:4095 trunc from 0x8f1e2c4 to 0x128f616 2024-09-13T05:02:47.125 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:47.127 INFO:teuthology.orchestra.run.smithi053.stdout:4096 punch from 0x62bad3 to 0x62d308, (0x1835 bytes) 2024-09-13T05:02:47.127 INFO:teuthology.orchestra.run.smithi053.stdout:4097 read 0x684e55 thru 0x689b26 (0x4cd2 bytes) 2024-09-13T05:02:47.140 INFO:teuthology.orchestra.run.smithi053.stdout:4098 trunc from 0x128f616 to 0xbd97683 2024-09-13T05:02:47.142 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:47.142 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:47.147 INFO:teuthology.orchestra.run.smithi053.stdout:4101 write 0x87958cd thru 0x879f4d7 (0x9c0b bytes) 2024-09-13T05:02:47.157 INFO:teuthology.orchestra.run.smithi053.stdout:4103 trunc from 0xbd97683 to 0x620b500 2024-09-13T05:02:47.160 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:47.160+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:47.163 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:47.163 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:47.170 INFO:teuthology.orchestra.run.smithi053.stdout:4104 trunc from 0x620b500 to 0x5468a6f 2024-09-13T05:02:47.176 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.176 INFO:teuthology.orchestra.run.smithi053.stdout:4107 punch from 0x1c8bbaf to 0x1c9ba92, (0xfee3 bytes) 2024-09-13T05:02:47.182 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:47.182 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:47.186 INFO:teuthology.orchestra.run.smithi053.stdout:4108 trunc from 0x5468a6f to 0xc1eb315 2024-09-13T05:02:47.195 INFO:teuthology.orchestra.run.smithi053.stdout:4109 trunc from 0xc1eb315 to 0xa297708 2024-09-13T05:02:47.201 INFO:teuthology.orchestra.run.smithi053.stdout:4110 read 0x7f1e876 thru 0x7f25a50 (0x71db bytes) 2024-09-13T05:02:47.201 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.201 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:47.202 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:47.202 INFO:teuthology.orchestra.run.smithi053.stdout:4112 write 0x89bc85b thru 0x89c096f (0x4115 bytes) 2024-09-13T05:02:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:4113 read 0x37b14bf thru 0x37b3610 (0x2152 bytes) 2024-09-13T05:02:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:4115 read 0x14d109f thru 0x14dda2f (0xc991 bytes) 2024-09-13T05:02:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.206 INFO:teuthology.orchestra.run.smithi053.stdout:4117 write 0x709659b thru 0x7097b5d (0x15c3 bytes) 2024-09-13T05:02:47.209 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.209 INFO:teuthology.orchestra.run.smithi053.stdout:4119 read 0x8abb2b4 thru 0x8ac0abe (0x580b bytes) 2024-09-13T05:02:47.209 INFO:teuthology.orchestra.run.smithi053.stdout:4122 punch from 0x8a42950 to 0x8a4624e, (0x38fe bytes) 2024-09-13T05:02:47.209 INFO:teuthology.orchestra.run.smithi053.stdout:4124 read 0x2186788 thru 0x2194f26 (0xe79f bytes) 2024-09-13T05:02:47.209 INFO:teuthology.orchestra.run.smithi053.stdout:4125 writesame 0x23b83c4 thru 0x23bc664 data_size 0x1(0x42a1 bytes) 2024-09-13T05:02:47.222 INFO:teuthology.orchestra.run.smithi053.stdout:4130 read 0x743fbaf thru 0x744e843 (0xec95 bytes) 2024-09-13T05:02:47.222 INFO:teuthology.orchestra.run.smithi053.stdout:4131 writesame 0x3207ac7 thru 0x3209ef6 data_size 0x18(0x2430 bytes) 2024-09-13T05:02:47.224 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:47.224 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:47.226 INFO:teuthology.orchestra.run.smithi053.stdout:4134 read 0x7a8523a thru 0x7a90513 (0xb2da bytes) 2024-09-13T05:02:47.226 INFO:teuthology.orchestra.run.smithi053.stdout:4135 read 0x86ffce0 thru 0x870c068 (0xc389 bytes) 2024-09-13T05:02:47.226 INFO:teuthology.orchestra.run.smithi053.stdout:4136 punch from 0x723bf63 to 0x724b51e, (0xf5bb bytes) 2024-09-13T05:02:47.227 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:47.227 INFO:teuthology.orchestra.run.smithi053.stdout:4138 write 0x8dab544 thru 0x8db28cc (0x7389 bytes) 2024-09-13T05:02:47.231 INFO:teuthology.orchestra.run.smithi053.stdout:4139 read 0x4373719 thru 0x4381216 (0xdafe bytes) 2024-09-13T05:02:47.231 INFO:teuthology.orchestra.run.smithi053.stdout:4140 write 0x3d3ea6c thru 0x3d4b177 (0xc70c bytes) 2024-09-13T05:02:47.236 INFO:teuthology.orchestra.run.smithi053.stdout:4142 write 0x27783f3 thru 0x277fb2f (0x773d bytes) 2024-09-13T05:02:47.237 INFO:teuthology.orchestra.run.smithi053.stdout:4143 read 0x1ae0835 thru 0x1ae5906 (0x50d2 bytes) 2024-09-13T05:02:47.243 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:47.243 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:47.246 INFO:teuthology.orchestra.run.smithi053.stdout:4144 writesame 0xbe36443 thru 0xbe3cf52 data_size 0x18(0x6b10 bytes) 2024-09-13T05:02:47.246 INFO:teuthology.orchestra.run.smithi053.stdout:4146 clone 24 order 21 su 8192 sc 11 2024-09-13T05:02:47.252 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:47.252 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:47.260 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:47.261 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:47.262 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:47.262 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:47.260+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:47.277 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:47.278 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:47.362 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:47.360+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:47.385 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:47.385 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:47.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:47.462 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:47.462+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:47.542 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:47.542 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:47.550 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:47.563 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:47.563+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:47.656 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:47.656 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:47.665 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:47.665 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:47.663+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:47.765 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:47.764+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:47.810 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:47.810 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:47.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:47.866 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:47.864+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:47.926 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:47.936 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:47.966 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:47.966+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:48.013 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:02:48.067 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:48.067+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:48.088 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone23 from 0xbe3cf53 (overlap 0x8eda76) to 0x80916a 2024-09-13T05:02:48.096 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.096 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.108 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:48.166 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-13T05:02:48.168 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:48.167+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:48.169 INFO:teuthology.orchestra.run.smithi053.stdout:4147 trunc from 0xbe3cf53 to 0xe00ae4c 2024-09-13T05:02:48.178 INFO:teuthology.orchestra.run.smithi053.stdout:4148 write 0x2f3e822 thru 0x2f408a5 (0x2084 bytes) 2024-09-13T05:02:48.185 INFO:teuthology.orchestra.run.smithi053.stdout:4152 writesame 0xbafc03f thru 0xbb0995f data_size 0x1(0xd921 bytes) 2024-09-13T05:02:48.198 INFO:teuthology.orchestra.run.smithi053.stdout:4153 write 0x7782ad4 thru 0x778d2b0 (0xa7dd bytes) 2024-09-13T05:02:48.207 INFO:teuthology.orchestra.run.smithi053.stdout:4155 read 0x3bfeee4 thru 0x3c05273 (0x6390 bytes) 2024-09-13T05:02:48.207 INFO:teuthology.orchestra.run.smithi053.stdout:4157 read 0xbf9af2f thru 0xbfa76ff (0xc7d1 bytes) 2024-09-13T05:02:48.208 INFO:teuthology.orchestra.run.smithi053.stdout:4159 write 0x400f77c thru 0x4019866 (0xa0eb bytes) 2024-09-13T05:02:48.217 INFO:teuthology.orchestra.run.smithi053.stdout:4160 punch from 0x1c10c36 to 0x1c121f5, (0x15bf bytes) 2024-09-13T05:02:48.228 INFO:teuthology.orchestra.run.smithi053.stdout:4161 trunc from 0xe00ae4c to 0x435dc4f 2024-09-13T05:02:48.231 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.231 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:48.250 INFO:teuthology.orchestra.run.smithi053.stdout:4164 read 0x1bede51 thru 0x1bf8353 (0xa503 bytes) 2024-09-13T05:02:48.250 INFO:teuthology.orchestra.run.smithi053.stdout:4165 clone 25 order 18 su 2048 sc 5 2024-09-13T05:02:48.286 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:48.286+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:48.345 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.346 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.354 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:48.395 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:48.394+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:48.472 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:48.471+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:48.496 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.496 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.504 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:48.573 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:48.572+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:48.607 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.607 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.615 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:48.700 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:48.699+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:48.758 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.759 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:48.809 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:48.808+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:48.875 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:48.875+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:48.910 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:48.910 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:48.919 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:48.977 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:48.975+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:49.023 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:49.023 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:49.032 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:49.078 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:49.076+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:49.107 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone24 from 0x435dc4f (overlap 0x435dc4f) to 0x0 2024-09-13T05:02:49.153 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:49.153 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:49.165 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:49.178 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:49.177+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:49.279 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:49.278+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:49.289 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:49.289 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:49.290 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:49.294 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-13T05:02:49.299 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:49.327 INFO:teuthology.orchestra.run.smithi053.stdout:4166 write 0x9e3b08b thru 0x9e42f89 (0x7eff bytes) 2024-09-13T05:02:49.335 INFO:teuthology.orchestra.run.smithi053.stdout:4167 punch from 0x9b51bec to 0x9b5d2d7, (0xb6eb bytes) 2024-09-13T05:02:49.335 INFO:teuthology.orchestra.run.smithi053.stdout:4168 read 0x173b686 thru 0x173e7d5 (0x3150 bytes) 2024-09-13T05:02:49.335 INFO:teuthology.orchestra.run.smithi053.stdout:4169 punch from 0x1d04bb6 to 0x1d10391, (0xb7db bytes) 2024-09-13T05:02:49.335 INFO:teuthology.orchestra.run.smithi053.stdout:4170 write 0x101da90 thru 0x1027a97 (0xa008 bytes) 2024-09-13T05:02:49.369 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.370 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.370 INFO:teuthology.orchestra.run.smithi053.stdout:4173 writesame 0x25ea16f thru 0x25f6e33 data_size 0x1f(0xccc5 bytes) 2024-09-13T05:02:49.380 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:49.380+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:49.394 INFO:teuthology.orchestra.run.smithi053.stdout:4174 writesame 0x6ac151 thru 0x6b372c data_size 0x26(0x75dc bytes) 2024-09-13T05:02:49.433 INFO:teuthology.orchestra.run.smithi053.stdout:4175 write 0x1c6ea4b thru 0x1c7c9f4 (0xdfaa bytes) 2024-09-13T05:02:49.438 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:49.439 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:49.452 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:49.456 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:49.456 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:49.461 INFO:teuthology.orchestra.run.smithi053.stdout:4177 trunc from 0x9e42f8a to 0xecd493a 2024-09-13T05:02:49.467 INFO:teuthology.orchestra.run.smithi053.stdout:4178 write 0x93f64c3 thru 0x9401729 (0xb267 bytes) 2024-09-13T05:02:49.474 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.474 INFO:teuthology.orchestra.run.smithi053.stdout:4182 write 0xdecac8 thru 0xdf82ca (0xb803 bytes) 2024-09-13T05:02:49.478 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:49.478 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:49.495 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:49.492+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:49.503 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:49.503 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:49.504 INFO:teuthology.orchestra.run.smithi053.stdout:4183 writesame 0xa29e22a thru 0xa2ac7bf data_size 0x1(0xe596 bytes) 2024-09-13T05:02:49.517 INFO:teuthology.orchestra.run.smithi053.stdout:4185 read 0x1c29582 thru 0x1c2f57a (0x5ff9 bytes) 2024-09-13T05:02:49.518 INFO:teuthology.orchestra.run.smithi053.stdout:4186 writesame 0x7e77e44 thru 0x7e7c5f8 data_size 0x1d(0x47b5 bytes) 2024-09-13T05:02:49.524 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:49.525 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:49.525 INFO:teuthology.orchestra.run.smithi053.stdout:4187 read 0xe69d5c7 thru 0xe6a3ca4 (0x66de bytes) 2024-09-13T05:02:49.525 INFO:teuthology.orchestra.run.smithi053.stdout:4190 read 0xdc5d3ca thru 0xdc68ba8 (0xb7df bytes) 2024-09-13T05:02:49.525 INFO:teuthology.orchestra.run.smithi053.stdout:4191 writesame 0x659690c thru 0x659ccaa data_size 0x1(0x639f bytes) 2024-09-13T05:02:49.535 INFO:teuthology.orchestra.run.smithi053.stdout:4192 read 0x2109e59 thru 0x2114bb4 (0xad5c bytes) 2024-09-13T05:02:49.536 INFO:teuthology.orchestra.run.smithi053.stdout:4194 punch from 0x1d8f659 to 0x1d90059, (0xa00 bytes) 2024-09-13T05:02:49.536 INFO:teuthology.orchestra.run.smithi053.stdout:4195 write 0xe52c6e4 thru 0xe539c69 (0xd586 bytes) 2024-09-13T05:02:49.545 INFO:teuthology.orchestra.run.smithi053.stdout:4196 write 0x7abd068 thru 0x7ac6f54 (0x9eed bytes) 2024-09-13T05:02:49.546 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:49.547 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:49.551 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.551 INFO:teuthology.orchestra.run.smithi053.stdout:4199 write 0xd30eaac thru 0xd31b760 (0xccb5 bytes) 2024-09-13T05:02:49.557 INFO:teuthology.orchestra.run.smithi053.stdout:4201 write 0xea7a026 thru 0xea86182 (0xc15d bytes) 2024-09-13T05:02:49.565 INFO:teuthology.orchestra.run.smithi053.stdout:4202 read 0xadc8e56 thru 0xadd8ac8 (0xfc73 bytes) 2024-09-13T05:02:49.571 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:49.571 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:49.575 INFO:teuthology.orchestra.run.smithi053.stdout:4204 trunc from 0xecd493a to 0x5845a28 2024-09-13T05:02:49.583 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:49.582+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:49.593 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:49.593 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:49.594 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.601 INFO:teuthology.orchestra.run.smithi053.stdout:4206 trunc from 0x5845a28 to 0x4f792 2024-09-13T05:02:49.621 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:49.621 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:49.624 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.624 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66005 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618950+0000", 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 3.1507999999999997e-05, 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3127000000000001e-05, 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66005 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618950+0000", 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618950+0000", 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618950+0000", 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618952+0000", 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.328e-06 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618958+0000", 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2310000000000001e-06 2024-09-13T05:02:49.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618963+0000", 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66006 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618967+0000", 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 1.4491999999999999e-05, 2024-09-13T05:02:49.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000128603, 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66006 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618967+0000", 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618967+0000", 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618967+0000", 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618969+0000", 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1299999999999999e-06 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618975+0000", 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0460000000000002e-06 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618980+0000", 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.639e-06 2024-09-13T05:02:49.631 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66007 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618984+0000", 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999976, 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00016893, 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66007 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618984+0000", 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618984+0000", 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618983+0000", 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618986+0000", 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0849999999999999e-06 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618991+0000", 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6609999999999997e-06 2024-09-13T05:02:49.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618996+0000", 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5650000000000003e-06 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66008 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619000+0000", 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999814, 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020978899999999999, 2024-09-13T05:02:49.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66008 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619000+0000", 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619000+0000", 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619000+0000", 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619002+0000", 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1789999999999998e-06 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619008+0000", 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0079999999999999e-06 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619013+0000", 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5820000000000003e-06 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.638 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66009 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619017+0000", 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999652, 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000249824, 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66009 2024-09-13T05:02:49.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619017+0000", 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619017+0000", 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619016+0000", 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619018+0000", 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9800000000000001e-06 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619024+0000", 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8150000000000002e-06 2024-09-13T05:02:49.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619029+0000", 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.685e-06 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66010 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619033+0000", 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999485, 2024-09-13T05:02:49.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029250599999999999, 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66010 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619033+0000", 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619033+0000", 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619033+0000", 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619035+0000", 2024-09-13T05:02:49.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619041+0000", 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9359999999999996e-06 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619045+0000", 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.583e-06 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66011 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619050+0000", 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999323, 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00033252100000000001, 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66011 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619050+0000", 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619050+0000", 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619049+0000", 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619051+0000", 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.902e-06 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619057+0000", 2024-09-13T05:02:49.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6980000000000003e-06 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619063+0000", 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6250000000000004e-06 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65934 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.617443+0000", 2024-09-13T05:02:49.649 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.0015382449999999999, 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019925199999999998, 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65934 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.617443+0000", 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.617443+0000", 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.617443+0000", 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.617446+0000", 2024-09-13T05:02:49.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2540000000000001e-06 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618493+0000", 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001047252 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618504+0000", 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0461e-05 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66048 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619067+0000", 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9999146, 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000423356, 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66048 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619067+0000", 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619067+0000", 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619066+0000", 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619069+0000", 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619075+0000", 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.981e-06 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619079+0000", 2024-09-13T05:02:49.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3710000000000002e-06 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65935 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618512+0000", 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00046941499999999999, 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001035408, 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65935 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618512+0000", 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618512+0000", 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618512+0000", 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618516+0000", 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4290000000000003e-06 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618525+0000", 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2739999999999996e-06 2024-09-13T05:02:49.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618531+0000", 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.711e-06 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66049 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619083+0000", 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998984, 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00052119000000000004, 2024-09-13T05:02:49.659 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66049 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619083+0000", 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619083+0000", 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619083+0000", 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619089+0000", 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4799999999999998e-06 2024-09-13T05:02:49.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619096+0000", 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619101+0000", 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.662 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65936 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618536+0000", 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00044573699999999998, 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0011224729999999999, 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65936 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618536+0000", 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618536+0000", 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618536+0000", 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618538+0000", 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3319999999999999e-06 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618544+0000", 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4169999999999997e-06 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618549+0000", 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7520000000000004e-06 2024-09-13T05:02:49.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66050 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619105+0000", 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998765, 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000609547, 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66050 2024-09-13T05:02:49.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619105+0000", 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619105+0000", 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619105+0000", 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619107+0000", 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0999999999999998e-06 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619113+0000", 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0859999999999998e-06 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619117+0000", 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2939999999999999e-06 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65937 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618554+0000", 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00042808100000000002, 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012180800000000001, 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.669 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65937 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618554+0000", 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618554+0000", 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618553+0000", 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618556+0000", 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3099999999999999e-06 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618562+0000", 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5069999999999997e-06 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618567+0000", 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7330000000000003e-06 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.672 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66051 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619121+0000", 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998603, 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00070569399999999998, 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66051 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.674 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619121+0000", 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619121+0000", 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619121+0000", 2024-09-13T05:02:49.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619123+0000", 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1660000000000001e-06 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619129+0000", 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1059999999999996e-06 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619134+0000", 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3769999999999998e-06 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65938 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618573+0000", 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00040866399999999999, 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001309621, 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65938 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618573+0000", 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618573+0000", 2024-09-13T05:02:49.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618573+0000", 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618575+0000", 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.39e-06 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618582+0000", 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.781e-06 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618586+0000", 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6419999999999998e-06 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66052 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619138+0000", 2024-09-13T05:02:49.680 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998436, 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00080319199999999999, 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66052 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619138+0000", 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619138+0000", 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619138+0000", 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619140+0000", 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.99e-06 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619146+0000", 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2349999999999996e-06 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619150+0000", 2024-09-13T05:02:49.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65939 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618591+0000", 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00039081400000000002, 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014047249999999999, 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65939 2024-09-13T05:02:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618591+0000", 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618591+0000", 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618591+0000", 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618593+0000", 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0260000000000002e-06 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618599+0000", 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2550000000000003e-06 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618603+0000", 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6210000000000003e-06 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66053 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619155+0000", 2024-09-13T05:02:49.688 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998269, 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00089759200000000001, 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66053 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619155+0000", 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619155+0000", 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619154+0000", 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619156+0000", 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1040000000000001e-06 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619162+0000", 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0090000000000004e-06 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619167+0000", 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4499999999999997e-06 2024-09-13T05:02:49.691 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65940 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618608+0000", 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00037377899999999997, 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0014971819999999999, 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.692 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65940 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618608+0000", 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618608+0000", 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618608+0000", 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618610+0000", 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.165e-06 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618616+0000", 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2190000000000001e-06 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618620+0000", 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5379999999999996e-06 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.695 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66054 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619171+0000", 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9998102, 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010075469999999999, 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66054 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619171+0000", 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619171+0000", 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619171+0000", 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619173+0000", 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1849999999999998e-06 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619179+0000", 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.066e-06 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619184+0000", 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5569999999999997e-06 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65941 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618625+0000", 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00035677400000000002, 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016069820000000001, 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65941 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.699 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618625+0000", 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618625+0000", 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618625+0000", 2024-09-13T05:02:49.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618627+0000", 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618633+0000", 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0340000000000002e-06 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618637+0000", 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6410000000000001e-06 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66055 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619188+0000", 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.999794, 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0010971469999999999, 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.702 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66055 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619188+0000", 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619188+0000", 2024-09-13T05:02:49.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619187+0000", 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619190+0000", 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4119999999999999e-06 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619195+0000", 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.541e-06 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619200+0000", 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6670000000000004e-06 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65942 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618642+0000", 2024-09-13T05:02:49.705 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000340129, 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001713589, 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65942 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618642+0000", 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618642+0000", 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618641+0000", 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618651+0000", 2024-09-13T05:02:49.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.5289999999999997e-06 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618658+0000", 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0199999999999997e-06 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618663+0000", 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.48e-06 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.708 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66056 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619204+0000", 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9997773, 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012047570000000001, 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66056 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.709 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619204+0000", 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619204+0000", 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619204+0000", 2024-09-13T05:02:49.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619206+0000", 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0839999999999999e-06 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619212+0000", 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1009999999999997e-06 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619217+0000", 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8559999999999998e-06 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65943 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618668+0000", 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00031394499999999999, 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001817613, 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.712 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65943 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618668+0000", 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618668+0000", 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618667+0000", 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618669+0000", 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.086e-06 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618675+0000", 2024-09-13T05:02:49.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0560000000000001e-06 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618680+0000", 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9620000000000001e-06 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66057 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619221+0000", 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9997602, 2024-09-13T05:02:49.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0013185409999999999, 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66057 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619221+0000", 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619221+0000", 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619221+0000", 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619223+0000", 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0980000000000001e-06 2024-09-13T05:02:49.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619229+0000", 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2169999999999999e-06 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619234+0000", 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6229999999999996e-06 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65979 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618685+0000", 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00029697000000000002, 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001908545, 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65979 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618685+0000", 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618685+0000", 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618684+0000", 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618686+0000", 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.119e-06 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618693+0000", 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1990000000000003e-06 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618697+0000", 2024-09-13T05:02:49.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5290000000000002e-06 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66058 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619238+0000", 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9997435, 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001420445, 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66058 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619238+0000", 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619238+0000", 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619238+0000", 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619240+0000", 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0169999999999999e-06 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619246+0000", 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2949999999999999e-06 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619251+0000", 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6700000000000002e-06 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65980 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618702+0000", 2024-09-13T05:02:49.725 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00027946600000000002, 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020106310000000001, 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65980 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618702+0000", 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618702+0000", 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618702+0000", 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618704+0000", 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618711+0000", 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7209999999999997e-06 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618716+0000", 2024-09-13T05:02:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.101e-06 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66059 3.e 3.dea3734e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619256+0000", 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9997253, 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001512289, 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66059 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619256+0000", 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619256+0000", 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619256+0000", 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619258+0000", 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.232e-06 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619264+0000", 2024-09-13T05:02:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0900000000000001e-06 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619269+0000", 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4320000000000001e-06 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65981 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618720+0000", 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00026137000000000003, 2024-09-13T05:02:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021142890000000001, 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65981 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618720+0000", 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618720+0000", 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618720+0000", 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618722+0000", 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3860000000000001e-06 2024-09-13T05:02:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618729+0000", 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2809999999999997e-06 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618733+0000", 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1339999999999997e-06 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66060 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619273+0000", 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9997082, 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016164580000000001, 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66060 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619273+0000", 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619273+0000", 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619273+0000", 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619276+0000", 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6340000000000001e-06 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619282+0000", 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1970000000000001e-06 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619287+0000", 2024-09-13T05:02:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.276e-06 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65982 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618737+0000", 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00024447800000000001, 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022146869999999999, 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65982 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618737+0000", 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618737+0000", 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618737+0000", 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618739+0000", 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1459999999999999e-06 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618745+0000", 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:02:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618750+0000", 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6820000000000002e-06 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66061 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619293+0000", 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9996886, 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0017118599999999999, 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66061 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619293+0000", 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619293+0000", 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619293+0000", 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619299+0000", 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1890000000000004e-06 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619326+0000", 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300999999999998e-05 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619332+0000", 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8309999999999997e-06 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65983 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618754+0000", 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000227624, 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002320899, 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65983 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618754+0000", 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618754+0000", 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618754+0000", 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618756+0000", 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9429999999999999e-06 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618762+0000", 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0190000000000003e-06 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618766+0000", 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6600000000000003e-06 2024-09-13T05:02:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66062 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619337+0000", 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9996443, 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.001793018, 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66062 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619337+0000", 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619337+0000", 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619337+0000", 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619339+0000", 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6690000000000002e-06 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619346+0000", 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.883e-06 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619351+0000", 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65984 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618771+0000", 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00021099399999999999, 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024130470000000002, 2024-09-13T05:02:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65984 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618771+0000", 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618771+0000", 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618770+0000", 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618772+0000", 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9539999999999998e-06 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618778+0000", 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1510000000000001e-06 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618783+0000", 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4399999999999998e-06 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66063 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619356+0000", 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9996262, 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0019028129999999999, 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66063 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619356+0000", 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619356+0000", 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619355+0000", 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619357+0000", 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1950000000000002e-06 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619364+0000", 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2400000000000004e-06 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619368+0000", 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:02:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65985 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618787+0000", 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00019443600000000001, 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025346470000000001, 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65985 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618787+0000", 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618787+0000", 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618787+0000", 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618789+0000", 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.967e-06 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618795+0000", 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9030000000000001e-06 2024-09-13T05:02:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618800+0000", 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7129999999999996e-06 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66064 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619373+0000", 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.999609, 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002010689, 2024-09-13T05:02:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66064 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619373+0000", 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619373+0000", 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619373+0000", 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619375+0000", 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0250000000000001e-06 2024-09-13T05:02:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619381+0000", 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9529999999999996e-06 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619385+0000", 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4630000000000003e-06 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65986 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618803+0000", 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00017819600000000001, 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0026325099999999998, 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65986 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618803+0000", 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618803+0000", 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618803+0000", 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618805+0000", 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1229999999999998e-06 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618811+0000", 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.19e-06 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618816+0000", 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8629999999999999e-06 2024-09-13T05:02:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66065 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619389+0000", 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995923, 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0020998060000000001, 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66065 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619389+0000", 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619389+0000", 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619389+0000", 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619391+0000", 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1940000000000001e-06 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619397+0000", 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8619999999999999e-06 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619402+0000", 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6779999999999999e-06 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65987 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618821+0000", 2024-09-13T05:02:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000161046, 2024-09-13T05:02:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002721195, 2024-09-13T05:02:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65987 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618821+0000", 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618821+0000", 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618820+0000", 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618822+0000", 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.058e-06 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618828+0000", 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0079999999999999e-06 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618833+0000", 2024-09-13T05:02:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.758e-06 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66066 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619406+0000", 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995756, 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022042419999999999, 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66066 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619406+0000", 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619406+0000", 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619406+0000", 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619408+0000", 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2579999999999998e-06 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619414+0000", 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1149999999999999e-06 2024-09-13T05:02:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619419+0000", 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65988 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618837+0000", 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00014438100000000001, 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002826416, 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65988 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618837+0000", 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618837+0000", 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618837+0000", 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618839+0000", 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0619999999999999e-06 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618845+0000", 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0990000000000004e-06 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618850+0000", 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7829999999999998e-06 2024-09-13T05:02:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66067 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619423+0000", 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995589, 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0022989730000000002, 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66067 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619423+0000", 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619423+0000", 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619422+0000", 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619424+0000", 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1260000000000001e-06 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619430+0000", 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8490000000000002e-06 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619435+0000", 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:65989 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618854+0000", 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.00012746200000000001, 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0029285090000000001, 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 65989 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618854+0000", 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618854+0000", 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618854+0000", 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618861+0000", 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8750000000000002e-06 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618867+0000", 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4749999999999998e-06 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618872+0000", 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.848e-06 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66068 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619439+0000", 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995422, 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0023999350000000002, 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66068 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619439+0000", 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619439+0000", 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619439+0000", 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619441+0000", 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2120000000000002e-06 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619447+0000", 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2090000000000002e-06 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619452+0000", 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6800000000000001e-06 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66001 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618877+0000", 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.000105179, 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0030158429999999998, 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66001 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618877+0000", 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618877+0000", 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618876+0000", 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618879+0000", 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4490000000000002e-06 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618885+0000", 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.212e-06 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618889+0000", 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7249999999999997e-06 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66069 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619457+0000", 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9995251, 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025012860000000001, 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66069 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619457+0000", 2024-09-13T05:02:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619457+0000", 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619456+0000", 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619458+0000", 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9889999999999999e-06 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619464+0000", 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0900000000000001e-06 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619472+0000", 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0630000000000006e-06 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66002 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618894+0000", 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 8.7540999999999994e-05, 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0031248320000000001, 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66002 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618894+0000", 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618894+0000", 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618894+0000", 2024-09-13T05:02:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618896+0000", 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618902+0000", 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3899999999999998e-06 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618907+0000", 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6430000000000003e-06 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66070 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619476+0000", 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.999505, 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0025976340000000001, 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66070 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619476+0000", 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619476+0000", 2024-09-13T05:02:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619476+0000", 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619478+0000", 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1500000000000002e-06 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619484+0000", 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9580000000000004e-06 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619489+0000", 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6090000000000003e-06 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66003 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618911+0000", 2024-09-13T05:02:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 7.0471999999999999e-05, 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.003216998, 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66003 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618911+0000", 2024-09-13T05:02:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618911+0000", 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618911+0000", 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618913+0000", 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9659999999999999e-06 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618919+0000", 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2700000000000001e-06 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618924+0000", 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6120000000000001e-06 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66071 3.f 3.5d3022cf (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619493+0000", 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9994888, 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.002696857, 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66071 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619493+0000", 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619493+0000", 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619493+0000", 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619495+0000", 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1909999999999999e-06 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619501+0000", 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619505+0000", 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66004 3.e 3.972e7d0e (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.618928+0000", 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 5.3560999999999998e-05, 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0033190279999999999, 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66004 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618928+0000", 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618928+0000", 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618928+0000", 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618930+0000", 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2079999999999999e-06 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618941+0000", 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0868e-05 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.618946+0000", 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9250000000000003e-06 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:66107 3.13 3.d7461cd3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-09-13T05:02:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:49.619510+0000", 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 4294967295.9994717, 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0028155279999999999, 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "queued for pg", 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 66107 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619510+0000", 2024-09-13T05:02:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619510+0000", 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619510+0000", 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619512+0000", 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619518+0000", 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9850000000000003e-06 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:49.619522+0000", 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-13T05:02:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 57 2024-09-13T05:02:49.822 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:49.824 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:49.824 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:49.822+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:49.825 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4210 read 0x1d065 thru 0x20b59 (0x3af5 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4211 punch from 0x4e2fb to 0x4f792, (0x1497 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4212 write 0x21845b9 thru 0x218d3ce (0x8e16 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4213 write 0x4f915aa thru 0x4f9243e (0xe95 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4215 trunc from 0x4f9243f to 0x14c9d52 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4217 punch from 0x1412f86 to 0x1421b67, (0xebe1 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4219 punch from 0xcf9f6b to 0xd08e78, (0xef0d bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4220 write 0x10d64b2 thru 0x10dc84a (0x6399 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4222 read 0xc91386 thru 0xc94afc (0x3777 bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4225 read 0xe8cc09 thru 0xe91253 (0x464b bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4226 read 0x7ed3c3 thru 0x7f0a9c (0x36da bytes) 2024-09-13T05:02:49.826 INFO:teuthology.orchestra.run.smithi053.stdout:4227 clone 26 order 22 su 16384 sc 14 2024-09-13T05:02:49.924 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:49.923+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:49.968 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:49.968 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:49.968 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:49.968 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:49.968 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:49.978 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:50.025 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:50.024+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:50.033 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:02:50.088 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:50.088 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:50.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:50.088 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.857412273000001, 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.089 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.856484164000001, 2024-09-13T05:02:50.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:50.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:02:50.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:50.100 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.855385755, 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:50.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:02:50.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.849325793999999, 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:50.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:02:50.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:02:50.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:50.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.849316799, 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:02:50.111 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:02:50.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.849308428000001, 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:02:50.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:02:50.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.849298406999999, 2024-09-13T05:02:50.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:02:50.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.849282683999999, 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:02:50.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.847380515000001, 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:02:50.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.134 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.847372556, 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:02:50.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.847364868, 2024-09-13T05:02:50.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:02:50.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.847354176, 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:02:50.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:02:50.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:02:50.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:50.148 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.847345935, 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:50.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:02:50.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.847322274, 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:50.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:02:50.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.845679815, 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.158 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:02:50.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:02:50.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.845671582000001, 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:02:50.162 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:02:50.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:02:50.166 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.845448853000001, 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.167 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:02:50.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:50.171 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.845436304, 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:50.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.175 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.843397077999999, 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.176 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:02:50.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:02:50.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 25.843387299, 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.185 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:02:50.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:50.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:02:50.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:02:50.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.191 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:50.191 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:50.191 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:50.192 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone25 from 0x14c9d52 (overlap 0x4f792) to 0x37a19 2024-09-13T05:02:50.192 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-13T05:02:50.193 INFO:teuthology.orchestra.run.smithi053.stdout:4228 trunc from 0x14c9d52 to 0x974cc5f 2024-09-13T05:02:50.193 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:50.191+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:50.193 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:50.205 INFO:teuthology.orchestra.run.smithi053.stdout:4231 trunc from 0x974cc5f to 0x413db0f 2024-09-13T05:02:50.211 INFO:teuthology.orchestra.run.smithi053.stdout:4232 read 0x12351e5 thru 0x1240522 (0xb33e bytes) 2024-09-13T05:02:50.211 INFO:teuthology.orchestra.run.smithi053.stdout:4234 read 0x169eee1 thru 0x16a83a6 (0x94c6 bytes) 2024-09-13T05:02:50.211 INFO:teuthology.orchestra.run.smithi053.stdout:4235 punch from 0x124ae16 to 0x1254b36, (0x9d20 bytes) 2024-09-13T05:02:50.211 INFO:teuthology.orchestra.run.smithi053.stdout:4236 write 0x87ab38 thru 0x885bef (0xb0b8 bytes) 2024-09-13T05:02:50.225 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:50.225 INFO:teuthology.orchestra.run.smithi053.stdout:4238 read 0x394eb17 thru 0x395ea41 (0xff2b bytes) 2024-09-13T05:02:50.228 INFO:teuthology.orchestra.run.smithi053.stdout:4239 trunc from 0x413db0f to 0x69c7f0c 2024-09-13T05:02:50.236 INFO:teuthology.orchestra.run.smithi053.stdout:4240 trunc from 0x69c7f0c to 0x3d70a39 2024-09-13T05:02:50.242 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:50.243 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:50.243 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T05:02:50.243 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T05:02:50.249 INFO:teuthology.orchestra.run.smithi053.stdout:4242 trunc from 0x3d70a39 to 0xad06834 2024-09-13T05:02:50.255 INFO:teuthology.orchestra.run.smithi053.stdout:4243 clone 27 order 21 su 8192 sc 12 2024-09-13T05:02:50.293 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:50.293+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:50.312 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:50.312 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:50.321 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:50.405 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:50.405+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:50.469 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:50.469 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:50.469 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:50.469 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:50.469 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:50.478 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:50.496 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:50.495+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 50, 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "01e1476b-1467-4e0e-b01c-0f8cdf8276fe", 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-13T05:01:45.075461+0000", 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-13T05:02:50.015382+0000", 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-13T05:01:55.160352+0000", 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-13T05:01:47.179750+0000", 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5804032, 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-13T05:02:50.535 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 4, 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-13T05:02:50.536 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "reef", 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.817858+0000", 2024-09-13T05:02:50.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:02:50.538 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:02:50.538 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:50.538 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:50.538 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:50.538 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:50.538 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:50.539 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:50.539 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:50.539 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:50.539 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:50.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:50.539 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-13T05:02:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-13T05:02:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-13T05:02:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-13T05:02:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-13T05:02:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:50.540 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:50.541 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:50.541 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:50.541 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:50.541 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:50.541 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:50.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.542 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "17", 2024-09-13T05:02:50.542 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:50.542 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:50.542 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:50.542 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:50.542 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 3, 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 17, 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:50.543 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:50.544 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:50.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:50.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:50.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:50.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:50.545 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:50.545 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:50.546 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:50.547 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:50.547 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:50.547 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:50.547 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:50.547 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:50.547 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:50.548 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:50.548 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:02:50.548 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:50.548 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:02:50.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.549 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:50.549 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2, 2024-09-13T05:02:50.549 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2, 2024-09-13T05:02:50.549 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1, 2024-09-13T05:02:50.549 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:02:50.549 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.550 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-13T05:02:50.551 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-13T05:02:50.551 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.981425+0000", 2024-09-13T05:02:50.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-13T05:02:50.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-13T05:02:50.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:50.551 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:50.552 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:50.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:50.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-13T05:02:50.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-13T05:02:50.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-13T05:02:50.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-13T05:02:50.553 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-13T05:02:50.554 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:50.554 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:50.554 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:50.554 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:50.554 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:50.554 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:50.555 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:50.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.555 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "16", 2024-09-13T05:02:50.555 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:50.555 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:50.555 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:50.556 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:50.557 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:50.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:50.557 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:50.557 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:50.557 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:50.557 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:50.558 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:50.558 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:50.558 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:50.558 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:50.558 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:50.558 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:50.559 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:50.559 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:50.559 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:50.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:50.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.559 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:50.560 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:50.560 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:50.560 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:50.560 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:50.561 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:50.561 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:50.562 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:50.562 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:50.562 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:50.562 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:50.562 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-13T05:02:50.562 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-13T05:02:50.563 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-13T05:02:50.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.563 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:50.563 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-13T05:02:50.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.563 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:50.564 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 8, 2024-09-13T05:02:50.564 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 8, 2024-09-13T05:02:50.564 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.25, 2024-09-13T05:02:50.564 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:02:50.564 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:02:50.564 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:50.565 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:50.565 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:50.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.565 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:02:50.566 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-13T05:02:50.566 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:02:05.508713+0000", 2024-09-13T05:02:50.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:02:50.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:02:50.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:02:50.566 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:02:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:02:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:02:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:02:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:02:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:02:50.567 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:02:50.568 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:02:50.568 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:02:50.568 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-13T05:02:50.568 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-13T05:02:50.568 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-13T05:02:50.568 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-13T05:02:50.569 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-13T05:02:50.569 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:02:50.569 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:02:50.569 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:02:50.569 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "50", 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:02:50.570 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:02:50.571 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:02:50.571 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:02:50.571 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:02:50.571 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 29, 2024-09-13T05:02:50.571 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 50, 2024-09-13T05:02:50.571 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:02:50.572 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:02:50.572 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:02:50.572 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:02:50.572 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:02:50.572 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:02:50.573 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:02:50.573 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:02:50.573 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:02:50.573 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:02:50.573 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:02:50.574 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:02:50.574 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:02:50.574 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:02:50.574 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:02:50.574 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:02:50.574 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:02:50.575 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:02:50.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:02:50.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.575 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:02:50.575 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:02:50.576 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:02:50.576 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:02:50.576 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:02:50.576 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:02:50.576 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:02:50.577 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:02:50.577 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:02:50.577 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:02:50.577 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:02:50.577 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:02:50.577 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:02:50.578 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:02:50.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.578 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:02:50.578 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.5, 2024-09-13T05:02:50.578 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.5, 2024-09-13T05:02:50.578 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1, 2024-09-13T05:02:50.579 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.5, 2024-09-13T05:02:50.579 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.5, 2024-09-13T05:02:50.579 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 1, 2024-09-13T05:02:50.579 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 1, 2024-09-13T05:02:50.579 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 1 2024-09-13T05:02:50.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.580 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:50.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-13T05:02:50.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.580 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "baf16a5c-4204-424b-8d8e-a5b1486aa052", 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.581 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.582 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.582 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.582 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.582 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.582 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.582 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.583 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.583 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-13T05:02:50.583 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.584 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-13T05:02:50.584 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.584 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.586 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-13T05:02:50.586 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.588 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.588 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.588 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.588 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-13T05:02:50.588 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1769628429", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1769628429", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1769628429", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1769628429", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f3df9a0f-3a9e-4665-9233-718bc0fe1223", 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.592 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-13T05:02:50.593 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.594 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.594 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-13T05:02:50.597 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/827277153", 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/827277153", 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/827277153", 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/827277153", 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.598 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1304a876-75c5-454b-a60b-8c9823c4aca8", 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.599 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 0, 2024-09-13T05:02:50.600 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.600 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.600 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.600 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.601 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-13T05:02:50.601 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.601 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.601 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.603 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-13T05:02:50.603 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.603 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.605 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.606 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.607 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/1334123504", 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/1334123504", 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/1334123504", 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/1334123504", 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.608 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "83668012-8ddb-4fe5-bb31-6e78d2723f12", 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.609 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6824", 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6825", 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.610 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6826", 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6827", 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6830", 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6831", 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6828", 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6829", 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6825/4134389313", 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6827/4134389313", 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6831/4134389313", 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6829/4134389313", 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f462232f-5144-4652-9139-9039d53e191a", 2024-09-13T05:02:50.614 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6808", 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6809", 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6810", 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6811", 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.617 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6815", 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6816", 2024-09-13T05:02:50.618 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6812", 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6813", 2024-09-13T05:02:50.619 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6809/3681797816", 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6811/3681797816", 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6816/3681797816", 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6813/3681797816", 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1d2d8607-f1d9-45e3-8486-3f5bb23b516b", 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.621 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6800", 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6801", 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6802", 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6803", 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6806", 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6807", 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6804", 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6805", 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6801/81482652", 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6803/81482652", 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6807/81482652", 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6805/81482652", 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "7fe0fb7d-0a40-4666-82a1-4e18f53b445c", 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.626 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6814", 2024-09-13T05:02:50.627 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6817", 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.628 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6818", 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6819", 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.629 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6822", 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6823", 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.630 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6820", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6821", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6817/576600432", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6819/576600432", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6823/576600432", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6821/576600432", 2024-09-13T05:02:50.631 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "811503be-abe3-4d64-a094-09ff675f3666", 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:02:50.632 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6824", 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6825", 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.633 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6826", 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6827", 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6830", 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6831", 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6828", 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6829", 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:02:50.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6825/896779674", 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6827/896779674", 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6831/896779674", 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6829/896779674", 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:50.637 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358655+0000", 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.638 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.343203+0000", 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.639 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.307081+0000", 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.315153+0000", 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.640 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.280902+0000", 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.641 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.302500+0000", 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.354685+0000", 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358223+0000", 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-13T05:02:50.643 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [], 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [], 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.645 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:50.648 INFO:tasks.thrashosds.thrasher:j is {'epoch': 50, 'fsid': '01e1476b-1467-4e0e-b01c-0f8cdf8276fe', 'created': '2024-09-13T05:01:45.075461+0000', 'modified': '2024-09-13T05:02:50.015382+0000', 'last_up_change': '2024-09-13T05:01:55.160352+0000', 'last_in_change': '2024-09-13T05:01:47.179750+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T05:01:55.817858+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T05:01:55.981425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T05:02:05.508713+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 29, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'baf16a5c-4204-424b-8d8e-a5b1486aa052', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1769628429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1769628429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1769628429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1769628429}]}, 'public_addr': '172.21.15.53:6817/1769628429', 'cluster_addr': '172.21.15.53:6819/1769628429', 'heartbeat_back_addr': '172.21.15.53:6823/1769628429', 'heartbeat_front_addr': '172.21.15.53:6821/1769628429', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f3df9a0f-3a9e-4665-9233-718bc0fe1223', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 827277153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 827277153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 827277153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 827277153}]}, 'public_addr': '172.21.15.53:6801/827277153', 'cluster_addr': '172.21.15.53:6803/827277153', 'heartbeat_back_addr': '172.21.15.53:6807/827277153', 'heartbeat_front_addr': '172.21.15.53:6805/827277153', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1304a876-75c5-454b-a60b-8c9823c4aca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 1334123504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 1334123504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 1334123504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 1334123504}]}, 'public_addr': '172.21.15.53:6809/1334123504', 'cluster_addr': '172.21.15.53:6811/1334123504', 'heartbeat_back_addr': '172.21.15.53:6815/1334123504', 'heartbeat_front_addr': '172.21.15.53:6813/1334123504', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '83668012-8ddb-4fe5-bb31-6e78d2723f12', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6824', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6825', 'nonce': 4134389313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6826', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6827', 'nonce': 4134389313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6830', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6831', 'nonce': 4134389313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6828', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6829', 'nonce': 4134389313}]}, 'public_addr': '172.21.15.53:6825/4134389313', 'cluster_addr': '172.21.15.53:6827/4134389313', 'heartbeat_back_addr': '172.21.15.53:6831/4134389313', 'heartbeat_front_addr': '172.21.15.53:6829/4134389313', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f462232f-5144-4652-9139-9039d53e191a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 3681797816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 3681797816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6815', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6816', 'nonce': 3681797816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 3681797816}]}, 'public_addr': '172.21.15.110:6809/3681797816', 'cluster_addr': '172.21.15.110:6811/3681797816', 'heartbeat_back_addr': '172.21.15.110:6816/3681797816', 'heartbeat_front_addr': '172.21.15.110:6813/3681797816', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1d2d8607-f1d9-45e3-8486-3f5bb23b516b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 81482652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 81482652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 81482652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 81482652}]}, 'public_addr': '172.21.15.110:6801/81482652', 'cluster_addr': '172.21.15.110:6803/81482652', 'heartbeat_back_addr': '172.21.15.110:6807/81482652', 'heartbeat_front_addr': '172.21.15.110:6805/81482652', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7fe0fb7d-0a40-4666-82a1-4e18f53b445c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 576600432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 576600432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 576600432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 576600432}]}, 'public_addr': '172.21.15.110:6817/576600432', 'cluster_addr': '172.21.15.110:6819/576600432', 'heartbeat_back_addr': '172.21.15.110:6823/576600432', 'heartbeat_front_addr': '172.21.15.110:6821/576600432', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '811503be-abe3-4d64-a094-09ff675f3666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6824', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6825', 'nonce': 896779674}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6826', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6827', 'nonce': 896779674}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6830', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6831', 'nonce': 896779674}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6828', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6829', 'nonce': 896779674}]}, 'public_addr': '172.21.15.110:6825/896779674', 'cluster_addr': '172.21.15.110:6827/896779674', 'heartbeat_back_addr': '172.21.15.110:6831/896779674', 'heartbeat_front_addr': '172.21.15.110:6829/896779674', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358655+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.343203+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.307081+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.315153+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.280902+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.302500+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.354685+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T05:02:50.648 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.649 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969873398000001, 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:02:50.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969857400999999, 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:02:50.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969840605000002, 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969823632000001, 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:02:50.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969807238000001, 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:02:50.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969790201999999, 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:02:50.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969756619000002, 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.677 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.96973749, 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:50.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:50.681 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969659621000002, 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969598875999999, 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:50.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969538662000001, 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:02:50.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:50.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969521335, 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:02:50.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.697 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969492162999998, 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969472479, 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:50.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:02:50.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:50.705 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969455046, 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:02:50.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:50.709 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969437404000001, 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:02:50.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:02:50.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969403228000001, 2024-09-13T05:02:50.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:02:50.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:02:50.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969377878, 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:02:50.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:50.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:02:50.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969360205000001, 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.725 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.969336000999999, 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:02:50.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:50.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.729 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.730 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:50.730 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:50.730 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:50.730 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:50.731 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:50.646+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:50.775 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:50.774+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:50.839 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:50.840 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:50.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:50.849 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:50.848+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:50.949 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:50.948+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:50.996 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:50.996 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:50.996 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:50.996 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:50.996 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:51.005 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:51.020 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:02:51.021 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:02:51.025 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:02:51.034 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-09-13T05:02:50.018927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1053371089,"num_objects":1400,"num_object_clones":347,"num_object_copies":2800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1400,"num_whiteouts":594,"num_read":81045,"num_read_kb":517509,"num_write":14578,"num_write_kb":302512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11447,"ondisk_log_size":11447,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":365884,"kb_used_data":39880,"kb_used_omap":12,"kb_used_meta":325875,"kb_avail":754608836,"statfs":{"total":773094113280,"available":772719448064,"internally_reserved":0,"allocated":40837120,"data_stored":418390476,"data_compressed":8765304,"data_compressed_allocated":29417472,"data_compressed_original":411746248,"omap_allocated":12743,"internal_metadata":333696569},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":160436272,"num_objects":386,"num_object_clones":13,"num_object_copies":772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":386,"num_whiteouts":319,"num_read":4852,"num_read_kb":21517,"num_write":1233,"num_write_kb":87895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.016173"},"pg_stats":[{"pgid":"3.1f","version":"47'230","reported_seq":1418,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.651034+0000","last_change":"2024-09-13T05:02:08.892195+0000","last_active":"2024-09-13T05:02:45.651034+0000","last_peered":"2024-09-13T05:02:45.651034+0000","last_clean":"2024-09-13T05:02:45.651034+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:02:45.651034+0000","last_undegraded":"2024-09-13T05:02:45.651034+0000","last_fullsized":"2024-09-13T05:02:45.651034+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001008562,"stat_sum":{"num_bytes":26875436,"num_objects":42,"num_object_clones":8,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":22,"num_read":1510,"num_read_kb":6329,"num_write":207,"num_write_kb":12185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"47'357","reported_seq":1247,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.911839+0000","last_change":"2024-09-13T05:02:08.935331+0000","last_active":"2024-09-13T05:02:45.911839+0000","last_peered":"2024-09-13T05:02:45.911839+0000","last_clean":"2024-09-13T05:02:45.911839+0000","last_became_active":"2024-09-13T05:02:06.884063+0000","last_became_peered":"2024-09-13T05:02:06.884063+0000","last_unstale":"2024-09-13T05:02:45.911839+0000","last_undegraded":"2024-09-13T05:02:45.911839+0000","last_fullsized":"2024-09-13T05:02:45.911839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":357,"log_dups_size":0,"ondisk_log_size":357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:54:39.216716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067803799999999995,"stat_sum":{"num_bytes":49215764,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":1763,"num_read_kb":12010,"num_write":497,"num_write_kb":17377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"47'282","reported_seq":828,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.518195+0000","last_change":"2024-09-13T05:02:08.887577+0000","last_active":"2024-09-13T05:02:45.518195+0000","last_peered":"2024-09-13T05:02:45.518195+0000","last_clean":"2024-09-13T05:02:45.518195+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:02:45.518195+0000","last_undegraded":"2024-09-13T05:02:45.518195+0000","last_fullsized":"2024-09-13T05:02:45.518195+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023315099999999999,"stat_sum":{"num_bytes":2058404,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":1974,"num_read_kb":2046,"num_write":356,"num_write_kb":1634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"47'370","reported_seq":3535,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.057930+0000","last_change":"2024-09-13T05:02:08.888247+0000","last_active":"2024-09-13T05:02:46.057930+0000","last_peered":"2024-09-13T05:02:46.057930+0000","last_clean":"2024-09-13T05:02:46.057930+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:02:46.057930+0000","last_undegraded":"2024-09-13T05:02:46.057930+0000","last_fullsized":"2024-09-13T05:02:46.057930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088904499999999998,"stat_sum":{"num_bytes":38094232,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":16,"num_read":4525,"num_read_kb":27899,"num_write":474,"num_write_kb":18334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"47'374","reported_seq":3482,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.585284+0000","last_change":"2024-09-13T05:02:08.891592+0000","last_active":"2024-09-13T05:02:45.585284+0000","last_peered":"2024-09-13T05:02:45.585284+0000","last_clean":"2024-09-13T05:02:45.585284+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:02:45.585284+0000","last_undegraded":"2024-09-13T05:02:45.585284+0000","last_fullsized":"2024-09-13T05:02:45.585284+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045096999999999997,"stat_sum":{"num_bytes":32010338,"num_objects":54,"num_object_clones":14,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":23,"num_read":4825,"num_read_kb":21478,"num_write":350,"num_write_kb":1993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"47'1327","reported_seq":3272,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.096235+0000","last_change":"2024-09-13T05:02:08.882149+0000","last_active":"2024-09-13T05:02:46.096235+0000","last_peered":"2024-09-13T05:02:46.096235+0000","last_clean":"2024-09-13T05:02:46.096235+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:02:46.096235+0000","last_undegraded":"2024-09-13T05:02:46.096235+0000","last_fullsized":"2024-09-13T05:02:46.096235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1327,"log_dups_size":0,"ondisk_log_size":1327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030432899999999999,"stat_sum":{"num_bytes":23949862,"num_objects":58,"num_object_clones":14,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":25,"num_read":8338,"num_read_kb":15950,"num_write":2293,"num_write_kb":9024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"47'188","reported_seq":429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.899980+0000","last_change":"2024-09-13T05:02:08.882945+0000","last_active":"2024-09-13T05:02:45.899980+0000","last_peered":"2024-09-13T05:02:45.899980+0000","last_clean":"2024-09-13T05:02:45.899980+0000","last_became_active":"2024-09-13T05:02:06.881244+0000","last_became_peered":"2024-09-13T05:02:06.881244+0000","last_unstale":"2024-09-13T05:02:45.899980+0000","last_undegraded":"2024-09-13T05:02:45.899980+0000","last_fullsized":"2024-09-13T05:02:45.899980+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":188,"log_dups_size":0,"ondisk_log_size":188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:21.706328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078365599999999998,"stat_sum":{"num_bytes":5670070,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":13,"num_read":255,"num_read_kb":6351,"num_write":180,"num_write_kb":5129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"47'215","reported_seq":1101,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.525808+0000","last_change":"2024-09-13T05:02:08.935674+0000","last_active":"2024-09-13T05:02:45.525808+0000","last_peered":"2024-09-13T05:02:45.525808+0000","last_clean":"2024-09-13T05:02:45.525808+0000","last_became_active":"2024-09-13T05:02:06.880998+0000","last_became_peered":"2024-09-13T05:02:06.880998+0000","last_unstale":"2024-09-13T05:02:45.525808+0000","last_undegraded":"2024-09-13T05:02:45.525808+0000","last_fullsized":"2024-09-13T05:02:45.525808+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":215,"log_dups_size":0,"ondisk_log_size":215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:51:16.614368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089100599999999996,"stat_sum":{"num_bytes":28914463,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":17,"num_read":926,"num_read_kb":7751,"num_write":319,"num_write_kb":19980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"47'207","reported_seq":992,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.014818+0000","last_change":"2024-09-13T05:02:08.892249+0000","last_active":"2024-09-13T05:02:46.014818+0000","last_peered":"2024-09-13T05:02:46.014818+0000","last_clean":"2024-09-13T05:02:46.014818+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:02:46.014818+0000","last_undegraded":"2024-09-13T05:02:46.014818+0000","last_fullsized":"2024-09-13T05:02:46.014818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00093220499999999997,"stat_sum":{"num_bytes":19024724,"num_objects":28,"num_object_clones":9,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":11,"num_read":1153,"num_read_kb":7703,"num_write":215,"num_write_kb":5100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"47'184","reported_seq":826,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.971150+0000","last_change":"2024-09-13T05:02:08.888457+0000","last_active":"2024-09-13T05:02:45.971150+0000","last_peered":"2024-09-13T05:02:45.971150+0000","last_clean":"2024-09-13T05:02:45.971150+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:02:45.971150+0000","last_undegraded":"2024-09-13T05:02:45.971150+0000","last_fullsized":"2024-09-13T05:02:45.971150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010543060000000001,"stat_sum":{"num_bytes":13320838,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":15,"num_read":643,"num_read_kb":9458,"num_write":167,"num_write_kb":3974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"47'414","reported_seq":1999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.017748+0000","last_change":"2024-09-13T05:02:08.934615+0000","last_active":"2024-09-13T05:02:46.017748+0000","last_peered":"2024-09-13T05:02:46.017748+0000","last_clean":"2024-09-13T05:02:46.017748+0000","last_became_active":"2024-09-13T05:02:06.883052+0000","last_became_peered":"2024-09-13T05:02:06.883052+0000","last_unstale":"2024-09-13T05:02:46.017748+0000","last_undegraded":"2024-09-13T05:02:46.017748+0000","last_fullsized":"2024-09-13T05:02:46.017748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:32:04.845507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042864699999999999,"stat_sum":{"num_bytes":48936094,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":2548,"num_read_kb":13729,"num_write":590,"num_write_kb":22355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"47'232","reported_seq":1138,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.947619+0000","last_change":"2024-09-13T05:02:08.882243+0000","last_active":"2024-09-13T05:02:45.947619+0000","last_peered":"2024-09-13T05:02:45.947619+0000","last_clean":"2024-09-13T05:02:45.947619+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:02:45.947619+0000","last_undegraded":"2024-09-13T05:02:45.947619+0000","last_fullsized":"2024-09-13T05:02:45.947619+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027033299999999999,"stat_sum":{"num_bytes":33695110,"num_objects":46,"num_object_clones":16,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":20,"num_read":947,"num_read_kb":3530,"num_write":212,"num_write_kb":12114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:43.034646+0000","last_change":"2024-09-13T05:01:59.394849+0000","last_active":"2024-09-13T05:02:43.034646+0000","last_peered":"2024-09-13T05:02:43.034646+0000","last_clean":"2024-09-13T05:02:43.034646+0000","last_became_active":"2024-09-13T05:01:57.178293+0000","last_became_peered":"2024-09-13T05:01:57.178293+0000","last_unstale":"2024-09-13T05:02:43.034646+0000","last_undegraded":"2024-09-13T05:02:43.034646+0000","last_fullsized":"2024-09-13T05:02:43.034646+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:03.945702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040222199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"47'225","reported_seq":1308,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.869534+0000","last_change":"2024-09-13T05:02:08.888247+0000","last_active":"2024-09-13T05:02:45.869534+0000","last_peered":"2024-09-13T05:02:45.869534+0000","last_clean":"2024-09-13T05:02:45.869534+0000","last_became_active":"2024-09-13T05:02:06.881356+0000","last_became_peered":"2024-09-13T05:02:06.881356+0000","last_unstale":"2024-09-13T05:02:45.869534+0000","last_undegraded":"2024-09-13T05:02:45.869534+0000","last_fullsized":"2024-09-13T05:02:45.869534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:37.032892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085252899999999998,"stat_sum":{"num_bytes":24950226,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":19,"num_read":1132,"num_read_kb":7977,"num_write":213,"num_write_kb":2749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:43.034646+0000","last_change":"2024-09-13T05:01:59.394844+0000","last_active":"2024-09-13T05:02:43.034646+0000","last_peered":"2024-09-13T05:02:43.034646+0000","last_clean":"2024-09-13T05:02:43.034646+0000","last_became_active":"2024-09-13T05:01:57.177751+0000","last_became_peered":"2024-09-13T05:01:57.177751+0000","last_unstale":"2024-09-13T05:02:43.034646+0000","last_undegraded":"2024-09-13T05:02:43.034646+0000","last_fullsized":"2024-09-13T05:02:43.034646+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:30:48.740698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039273500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"47'623","reported_seq":2475,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.860736+0000","last_change":"2024-09-13T05:02:08.883472+0000","last_active":"2024-09-13T05:02:45.860736+0000","last_peered":"2024-09-13T05:02:45.860736+0000","last_clean":"2024-09-13T05:02:45.860736+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:02:45.860736+0000","last_undegraded":"2024-09-13T05:02:45.860736+0000","last_fullsized":"2024-09-13T05:02:45.860736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012097049999999999,"stat_sum":{"num_bytes":37655745,"num_objects":64,"num_object_clones":19,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":24,"num_read":4524,"num_read_kb":26482,"num_write":842,"num_write_kb":7451,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:42.996612+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:02:42.996612+0000","last_peered":"2024-09-13T05:02:42.996612+0000","last_clean":"2024-09-13T05:02:42.996612+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:02:42.996612+0000","last_undegraded":"2024-09-13T05:02:42.996612+0000","last_fullsized":"2024-09-13T05:02:42.996612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"47'311","reported_seq":829,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.777594+0000","last_change":"2024-09-13T05:02:08.935465+0000","last_active":"2024-09-13T05:02:45.777594+0000","last_peered":"2024-09-13T05:02:45.777594+0000","last_clean":"2024-09-13T05:02:45.777594+0000","last_became_active":"2024-09-13T05:02:06.884640+0000","last_became_peered":"2024-09-13T05:02:06.884640+0000","last_unstale":"2024-09-13T05:02:45.777594+0000","last_undegraded":"2024-09-13T05:02:45.777594+0000","last_fullsized":"2024-09-13T05:02:45.777594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":311,"log_dups_size":0,"ondisk_log_size":311,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:51.793989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075041500000000004,"stat_sum":{"num_bytes":77784603,"num_objects":53,"num_object_clones":12,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":25,"num_read":762,"num_read_kb":22058,"num_write":305,"num_write_kb":22922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:43.031144+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:02:43.031144+0000","last_peered":"2024-09-13T05:02:43.031144+0000","last_clean":"2024-09-13T05:02:43.031144+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:02:43.031144+0000","last_undegraded":"2024-09-13T05:02:43.031144+0000","last_fullsized":"2024-09-13T05:02:43.031144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"47'240","reported_seq":1416,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.732922+0000","last_change":"2024-09-13T05:02:08.891922+0000","last_active":"2024-09-13T05:02:45.732922+0000","last_peered":"2024-09-13T05:02:45.732922+0000","last_clean":"2024-09-13T05:02:45.732922+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:02:45.732922+0000","last_undegraded":"2024-09-13T05:02:45.732922+0000","last_fullsized":"2024-09-13T05:02:45.732922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":240,"log_dups_size":0,"ondisk_log_size":240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075461600000000005,"stat_sum":{"num_bytes":30540358,"num_objects":39,"num_object_clones":10,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":1973,"num_read_kb":15563,"num_write":223,"num_write_kb":9391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"47'296","reported_seq":1801,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.030574+0000","last_change":"2024-09-13T05:02:08.891590+0000","last_active":"2024-09-13T05:02:46.030574+0000","last_peered":"2024-09-13T05:02:46.030574+0000","last_clean":"2024-09-13T05:02:46.030574+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:02:46.030574+0000","last_undegraded":"2024-09-13T05:02:46.030574+0000","last_fullsized":"2024-09-13T05:02:46.030574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071068800000000001,"stat_sum":{"num_bytes":45979152,"num_objects":41,"num_object_clones":8,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":2306,"num_read_kb":21314,"num_write":341,"num_write_kb":4509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":191,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:42.998921+0000","last_change":"2024-09-13T05:01:57.175202+0000","last_active":"2024-09-13T05:02:42.998921+0000","last_peered":"2024-09-13T05:02:42.998921+0000","last_clean":"2024-09-13T05:02:42.998921+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:02:42.998921+0000","last_undegraded":"2024-09-13T05:02:42.998921+0000","last_fullsized":"2024-09-13T05:02:42.998921+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:24:29.733421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":47,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:43.024443+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:02:43.024443+0000","last_peered":"2024-09-13T05:02:43.024443+0000","last_clean":"2024-09-13T05:02:43.024443+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:02:43.024443+0000","last_undegraded":"2024-09-13T05:02:43.024443+0000","last_fullsized":"2024-09-13T05:02:43.024443+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"47'419","reported_seq":3102,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.636616+0000","last_change":"2024-09-13T05:02:08.883195+0000","last_active":"2024-09-13T05:02:45.636616+0000","last_peered":"2024-09-13T05:02:45.636616+0000","last_clean":"2024-09-13T05:02:45.636616+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:02:45.636616+0000","last_undegraded":"2024-09-13T05:02:45.636616+0000","last_fullsized":"2024-09-13T05:02:45.636616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":419,"log_dups_size":0,"ondisk_log_size":419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097658399999999996,"stat_sum":{"num_bytes":55871125,"num_objects":56,"num_object_clones":14,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":28,"num_read":4317,"num_read_kb":36743,"num_write":449,"num_write_kb":11440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":48,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:42.999000+0000","last_change":"2024-09-13T05:01:59.195603+0000","last_active":"2024-09-13T05:02:42.999000+0000","last_peered":"2024-09-13T05:02:42.999000+0000","last_clean":"2024-09-13T05:02:42.999000+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:02:42.999000+0000","last_undegraded":"2024-09-13T05:02:42.999000+0000","last_fullsized":"2024-09-13T05:02:42.999000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:04:02.276500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"47'296","reported_seq":2144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.900478+0000","last_change":"2024-09-13T05:02:08.882467+0000","last_active":"2024-09-13T05:02:45.900478+0000","last_peered":"2024-09-13T05:02:45.900478+0000","last_clean":"2024-09-13T05:02:45.900478+0000","last_became_active":"2024-09-13T05:02:06.880673+0000","last_became_peered":"2024-09-13T05:02:06.880673+0000","last_unstale":"2024-09-13T05:02:45.900478+0000","last_undegraded":"2024-09-13T05:02:45.900478+0000","last_fullsized":"2024-09-13T05:02:45.900478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:45:24.929086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048698900000000001,"stat_sum":{"num_bytes":46642183,"num_objects":40,"num_object_clones":8,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":14,"num_read":2928,"num_read_kb":31671,"num_write":281,"num_write_kb":9981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:43.009980+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:02:43.009980+0000","last_peered":"2024-09-13T05:02:43.009980+0000","last_clean":"2024-09-13T05:02:43.009980+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:02:43.009980+0000","last_undegraded":"2024-09-13T05:02:43.009980+0000","last_fullsized":"2024-09-13T05:02:43.009980+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"47'969","reported_seq":1385,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.975314+0000","last_change":"2024-09-13T05:02:08.934991+0000","last_active":"2024-09-13T05:02:45.975314+0000","last_peered":"2024-09-13T05:02:45.975314+0000","last_clean":"2024-09-13T05:02:45.975314+0000","last_became_active":"2024-09-13T05:02:06.882765+0000","last_became_peered":"2024-09-13T05:02:06.882765+0000","last_unstale":"2024-09-13T05:02:45.975314+0000","last_undegraded":"2024-09-13T05:02:45.975314+0000","last_fullsized":"2024-09-13T05:02:45.975314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":969,"log_dups_size":0,"ondisk_log_size":969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:21.053116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045448500000000002,"stat_sum":{"num_bytes":5717605,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":20,"num_read":4872,"num_read_kb":9806,"num_write":1645,"num_write_kb":4838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-13T05:02:42.998933+0000","last_change":"2024-09-13T05:01:59.195613+0000","last_active":"2024-09-13T05:02:42.998933+0000","last_peered":"2024-09-13T05:02:42.998933+0000","last_clean":"2024-09-13T05:02:42.998933+0000","last_became_active":"2024-09-13T05:01:57.172867+0000","last_became_peered":"2024-09-13T05:01:57.172867+0000","last_unstale":"2024-09-13T05:02:42.998933+0000","last_undegraded":"2024-09-13T05:02:42.998933+0000","last_fullsized":"2024-09-13T05:02:42.998933+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:03:56.676979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033540799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"47'292","reported_seq":2361,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.776826+0000","last_change":"2024-09-13T05:02:08.935586+0000","last_active":"2024-09-13T05:02:45.776826+0000","last_peered":"2024-09-13T05:02:45.776826+0000","last_clean":"2024-09-13T05:02:45.776826+0000","last_became_active":"2024-09-13T05:02:06.881316+0000","last_became_peered":"2024-09-13T05:02:06.881316+0000","last_unstale":"2024-09-13T05:02:45.776826+0000","last_undegraded":"2024-09-13T05:02:45.776826+0000","last_fullsized":"2024-09-13T05:02:45.776826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:07.292949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084766500000000005,"stat_sum":{"num_bytes":42757581,"num_objects":53,"num_object_clones":15,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":19,"num_read":2316,"num_read_kb":21490,"num_write":358,"num_write_kb":9784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"47'251","reported_seq":2049,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.437317+0000","last_change":"2024-09-13T05:02:08.882379+0000","last_active":"2024-09-13T05:02:45.437317+0000","last_peered":"2024-09-13T05:02:45.437317+0000","last_clean":"2024-09-13T05:02:45.437317+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:02:45.437317+0000","last_undegraded":"2024-09-13T05:02:45.437317+0000","last_fullsized":"2024-09-13T05:02:45.437317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":251,"log_dups_size":0,"ondisk_log_size":251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039834799999999997,"stat_sum":{"num_bytes":42274114,"num_objects":42,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":17,"num_read":2216,"num_read_kb":20820,"num_write":328,"num_write_kb":8800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"47'301","reported_seq":1189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.777438+0000","last_change":"2024-09-13T05:02:08.934869+0000","last_active":"2024-09-13T05:02:45.777438+0000","last_peered":"2024-09-13T05:02:45.777438+0000","last_clean":"2024-09-13T05:02:45.777438+0000","last_became_active":"2024-09-13T05:02:06.887529+0000","last_became_peered":"2024-09-13T05:02:06.887529+0000","last_unstale":"2024-09-13T05:02:45.777438+0000","last_undegraded":"2024-09-13T05:02:45.777438+0000","last_fullsized":"2024-09-13T05:02:45.777438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":301,"log_dups_size":0,"ondisk_log_size":301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:43.153535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050065899999999998,"stat_sum":{"num_bytes":34759452,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":27,"num_read":1218,"num_read_kb":6340,"num_write":321,"num_write_kb":12575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"47'715","reported_seq":1953,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.530439+0000","last_change":"2024-09-13T05:02:08.934984+0000","last_active":"2024-09-13T05:02:45.530439+0000","last_peered":"2024-09-13T05:02:45.530439+0000","last_clean":"2024-09-13T05:02:45.530439+0000","last_became_active":"2024-09-13T05:02:06.881094+0000","last_became_peered":"2024-09-13T05:02:06.881094+0000","last_unstale":"2024-09-13T05:02:45.530439+0000","last_undegraded":"2024-09-13T05:02:45.530439+0000","last_fullsized":"2024-09-13T05:02:45.530439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:02:56.840152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046295800000000002,"stat_sum":{"num_bytes":11823113,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":4222,"num_read_kb":8859,"num_write":1089,"num_write_kb":2811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"47'195","reported_seq":706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.004723+0000","last_change":"2024-09-13T05:02:08.887634+0000","last_active":"2024-09-13T05:02:46.004723+0000","last_peered":"2024-09-13T05:02:46.004723+0000","last_clean":"2024-09-13T05:02:46.004723+0000","last_became_active":"2024-09-13T05:02:06.879695+0000","last_became_peered":"2024-09-13T05:02:06.879695+0000","last_unstale":"2024-09-13T05:02:46.004723+0000","last_undegraded":"2024-09-13T05:02:46.004723+0000","last_fullsized":"2024-09-13T05:02:46.004723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:23.915465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028084900000000001,"stat_sum":{"num_bytes":7920319,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":17,"num_read":915,"num_read_kb":3858,"num_write":174,"num_write_kb":1372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"47'302","reported_seq":3429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:46.093861+0000","last_change":"2024-09-13T05:02:08.935026+0000","last_active":"2024-09-13T05:02:46.093861+0000","last_peered":"2024-09-13T05:02:46.093861+0000","last_clean":"2024-09-13T05:02:46.093861+0000","last_became_active":"2024-09-13T05:02:06.885421+0000","last_became_peered":"2024-09-13T05:02:06.885421+0000","last_unstale":"2024-09-13T05:02:46.093861+0000","last_undegraded":"2024-09-13T05:02:46.093861+0000","last_fullsized":"2024-09-13T05:02:46.093861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:20:39.514148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071617299999999996,"stat_sum":{"num_bytes":23896824,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":4040,"num_read_kb":24556,"num_write":371,"num_write_kb":2760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"47'291","reported_seq":2018,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.528123+0000","last_change":"2024-09-13T05:02:08.934818+0000","last_active":"2024-09-13T05:02:45.528123+0000","last_peered":"2024-09-13T05:02:45.528123+0000","last_clean":"2024-09-13T05:02:45.528123+0000","last_became_active":"2024-09-13T05:02:06.881683+0000","last_became_peered":"2024-09-13T05:02:06.881683+0000","last_unstale":"2024-09-13T05:02:45.528123+0000","last_undegraded":"2024-09-13T05:02:45.528123+0000","last_fullsized":"2024-09-13T05:02:45.528123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":291,"log_dups_size":0,"ondisk_log_size":291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:25:51.679049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057826299999999998,"stat_sum":{"num_bytes":63838217,"num_objects":49,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":20,"num_read":2149,"num_read_kb":50629,"num_write":371,"num_write_kb":27513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"47'211","reported_seq":1539,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.840056+0000","last_change":"2024-09-13T05:02:08.934811+0000","last_active":"2024-09-13T05:02:45.840056+0000","last_peered":"2024-09-13T05:02:45.840056+0000","last_clean":"2024-09-13T05:02:45.840056+0000","last_became_active":"2024-09-13T05:02:06.884741+0000","last_became_peered":"2024-09-13T05:02:06.884741+0000","last_unstale":"2024-09-13T05:02:45.840056+0000","last_undegraded":"2024-09-13T05:02:45.840056+0000","last_fullsized":"2024-09-13T05:02:45.840056+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":211,"log_dups_size":0,"ondisk_log_size":211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:29:29.184441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053391699999999996,"stat_sum":{"num_bytes":36145853,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":18,"num_read":1767,"num_read_kb":11813,"num_write":193,"num_write_kb":5260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"47'262","reported_seq":1238,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.346933+0000","last_change":"2024-09-13T05:02:08.934690+0000","last_active":"2024-09-13T05:02:45.346933+0000","last_peered":"2024-09-13T05:02:45.346933+0000","last_clean":"2024-09-13T05:02:45.346933+0000","last_became_active":"2024-09-13T05:02:06.884642+0000","last_became_peered":"2024-09-13T05:02:06.884642+0000","last_unstale":"2024-09-13T05:02:45.346933+0000","last_undegraded":"2024-09-13T05:02:45.346933+0000","last_fullsized":"2024-09-13T05:02:45.346933+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":262,"log_dups_size":0,"ondisk_log_size":262,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:16:18.856603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000322436,"stat_sum":{"num_bytes":24930631,"num_objects":36,"num_object_clones":9,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":18,"num_read":1211,"num_read_kb":7382,"num_write":335,"num_write_kb":3426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"47'293","reported_seq":2712,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.712500+0000","last_change":"2024-09-13T05:02:08.934351+0000","last_active":"2024-09-13T05:02:45.712500+0000","last_peered":"2024-09-13T05:02:45.712500+0000","last_clean":"2024-09-13T05:02:45.712500+0000","last_became_active":"2024-09-13T05:02:06.884670+0000","last_became_peered":"2024-09-13T05:02:06.884670+0000","last_unstale":"2024-09-13T05:02:45.712500+0000","last_undegraded":"2024-09-13T05:02:45.712500+0000","last_fullsized":"2024-09-13T05:02:45.712500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:15.372629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028879200000000001,"stat_sum":{"num_bytes":30068903,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":18,"num_read":3599,"num_read_kb":9586,"num_write":273,"num_write_kb":1460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"47'274","reported_seq":3150,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.732705+0000","last_change":"2024-09-13T05:02:08.891841+0000","last_active":"2024-09-13T05:02:45.732705+0000","last_peered":"2024-09-13T05:02:45.732705+0000","last_clean":"2024-09-13T05:02:45.732705+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:02:45.732705+0000","last_undegraded":"2024-09-13T05:02:45.732705+0000","last_fullsized":"2024-09-13T05:02:45.732705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068869099999999996,"stat_sum":{"num_bytes":47546735,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":17,"num_read":3731,"num_read_kb":34990,"num_write":270,"num_write_kb":17302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"47'181","reported_seq":1486,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-13T05:02:45.430705+0000","last_change":"2024-09-13T05:02:08.934570+0000","last_active":"2024-09-13T05:02:45.430705+0000","last_peered":"2024-09-13T05:02:45.430705+0000","last_clean":"2024-09-13T05:02:45.430705+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:02:45.430705+0000","last_undegraded":"2024-09-13T05:02:45.430705+0000","last_fullsized":"2024-09-13T05:02:45.430705+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042496100000000001,"stat_sum":{"num_bytes":39912628,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":1324,"num_read_kb":11238,"num_write":175,"num_write_kb":4551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1052780702,"num_objects":1396,"num_object_clones":347,"num_object_copies":2792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1396,"num_whiteouts":594,"num_read":80929,"num_read_kb":517409,"num_write":14417,"num_write_kb":300094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":26,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36749312,"data_stored":415811848,"data_compressed":8861436,"data_compressed_allocated":29499392,"data_compressed_original":410988488,"omap_allocated":0,"internal_metadata":0},"log_size":11327,"ondisk_log_size":11327,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47036,"kb_used_data":7724,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94324804,"statfs":{"total":96636764160,"available":96588599296,"internally_reserved":0,"allocated":7909376,"data_stored":100992986,"data_compressed":1762023,"data_compressed_allocated":6184960,"data_compressed_original":99970946,"omap_allocated":1589,"internal_metadata":40237515},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36972,"kb_used_data":1760,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334868,"statfs":{"total":96636764160,"available":96598904832,"internally_reserved":0,"allocated":1802240,"data_stored":11558612,"data_compressed":199128,"data_compressed_allocated":921600,"data_compressed_original":11104256,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42632,"kb_used_data":3324,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94329208,"statfs":{"total":96636764160,"available":96593108992,"internally_reserved":0,"allocated":3403776,"data_stored":28528637,"data_compressed":805208,"data_compressed_allocated":2174976,"data_compressed_original":27810029,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56600,"kb_used_data":7440,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94315240,"statfs":{"total":96636764160,"available":96578805760,"internally_reserved":0,"allocated":7618560,"data_stored":81304044,"data_compressed":1702461,"data_compressed_allocated":5611520,"data_compressed_original":80101999,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]}]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51284,"kb_used_data":6188,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94320556,"statfs":{"total":96636764160,"available":96584249344,"internally_reserved":0,"allocated":6336512,"data_stored":65750810,"data_compressed":1352262,"data_compressed_allocated":4677632,"data_compressed_original":64798527,"omap_allocated":1592,"internal_metadata":46135752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38064,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333776,"statfs":{"total":96636764160,"available":96597786624,"internally_reserved":0,"allocated":2920448,"data_stored":26971683,"data_compressed":698393,"data_compressed_allocated":1933312,"data_compressed_original":26419768,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48212,"kb_used_data":4812,"kb_used_omap":1,"kb_used_meta":43390,"kb_avail":94323628,"statfs":{"total":96636764160,"available":96587395072,"internally_reserved":0,"allocated":4927488,"data_stored":47231578,"data_compressed":1023688,"data_compressed_allocated":3645440,"data_compressed_original":46486327,"omap_allocated":1594,"internal_metadata":44431814},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45084,"kb_used_data":5780,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94326756,"statfs":{"total":96636764160,"available":96590598144,"internally_reserved":0,"allocated":5918720,"data_stored":56052126,"data_compressed":1222141,"data_compressed_allocated":4268032,"data_compressed_original":55054396,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.186}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5496832,"data_stored":55970191,"data_compressed":1281140,"data_compressed_allocated":4333568,"data_compressed_original":55185468,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4374528,"data_stored":46461395,"data_compressed":1026746,"data_compressed_allocated":3624960,"data_compressed_original":45949751,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2383872,"data_stored":26734972,"data_compressed":698393,"data_compressed_allocated":1933312,"data_compressed_original":26419768,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":5844992,"data_stored":65601983,"data_compressed":1352570,"data_compressed_allocated":4689920,"data_compressed_original":64864063,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":81089908,"data_compressed":1702496,"data_compressed_allocated":5615616,"data_compressed_original":80122479,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2932736,"data_stored":28373169,"data_compressed":835976,"data_compressed_allocated":2211840,"data_compressed_original":27875565,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1273856,"data_stored":11344405,"data_compressed":199163,"data_compressed_allocated":925696,"data_compressed_original":11124736,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7352320,"data_stored":100235825,"data_compressed":1764952,"data_compressed_allocated":6164480,"data_compressed_original":99446658,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:02:51.037 INFO:tasks.thrashosds.thrasher:Setting 3.f to [3, 5, 7, 2] 2024-09-13T05:02:51.037 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.f', '3', '5', '7', '2'] 2024-09-13T05:02:51.037 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.f 3 5 7 2 2024-09-13T05:02:51.050 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:51.049+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:51.124 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone26 intact 2024-09-13T05:02:51.139 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775226642, 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775203598999999, 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.145 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:51.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775186267, 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:02:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775169722999999, 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:51.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:02:51.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775152598, 2024-09-13T05:02:51.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:02:51.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.77513637, 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:02:51.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775119559, 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:51.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:02:51.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775102941, 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775086363, 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:51.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.180 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775050008999999, 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.181 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:02:51.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:02:51.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:51.185 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775031065, 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:51.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:02:51.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:02:51.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.775014431000001, 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:02:51.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:02:51.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.774988430000001, 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.199 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.774956490999999, 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:02:51.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:02:51.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.774937703999999, 2024-09-13T05:02:51.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:02:51.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.211 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.774921583999999, 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:02:51.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:02:51.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:51.216 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.774899523, 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:02:51.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:02:51.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.774882558, 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.221 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:02:51.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.225 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.462848159, 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:51.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.228 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:02:51.229 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.462829051, 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:51.230 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.231 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.232 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:02:51.233 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.234 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:51.239 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:51.239 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:51.235+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:51.319 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-13T05:02:51.323 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:51.323 INFO:teuthology.orchestra.run.smithi053.stdout:4246 read 0x71a9543 thru 0x71b38f0 (0xa3ae bytes) 2024-09-13T05:02:51.332 INFO:teuthology.orchestra.run.smithi053.stdout:4247 trunc from 0xad06834 to 0x35507b6 2024-09-13T05:02:51.337 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:51.336+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:51.362 INFO:teuthology.orchestra.run.smithi053.stdout:4248 read 0x7624cd thru 0x7704ab (0xdfdf bytes) 2024-09-13T05:02:51.374 INFO:teuthology.orchestra.run.smithi053.stdout:4250 trunc from 0x35507b6 to 0xce45ab6 2024-09-13T05:02:51.378 INFO:teuthology.orchestra.run.smithi053.stdout:4252 write 0x451574c thru 0x451f9bc (0xa271 bytes) 2024-09-13T05:02:51.387 INFO:teuthology.orchestra.run.smithi053.stdout:4253 write 0xdfd5a09 thru 0xdfe37d2 (0xddca bytes) 2024-09-13T05:02:51.396 INFO:teuthology.orchestra.run.smithi053.stdout:4254 write 0x616c638 thru 0x6178676 (0xc03f bytes) 2024-09-13T05:02:51.406 INFO:teuthology.orchestra.run.smithi053.stdout:4255 read 0x7f8a405 thru 0x7f90d94 (0x6990 bytes) 2024-09-13T05:02:51.406 INFO:teuthology.orchestra.run.smithi053.stdout:4256 punch from 0x4132bd6 to 0x413d978, (0xada2 bytes) 2024-09-13T05:02:51.406 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:51.406 INFO:teuthology.orchestra.run.smithi053.stdout:4258 clone 28 order 22 su 2097152 sc 11 2024-09-13T05:02:51.409 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:51.409 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:51.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:51.459 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:51.458+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:51.538 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:51.538+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:51.564 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:51.565 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:51.565 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:51.565 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:51.565 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:51.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:51.639 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:51.639+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:51.722 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:51.723 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:51.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:51.723 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:51.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.723 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:02:51.724 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:51.724 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.742855204000001, 2024-09-13T05:02:51.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:02:51.724 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.724 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.724 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.725 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.725 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:51.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:02:51.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:02:51.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:02:51.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:02:51.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:02:51.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:02:51.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:02:51.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:02:51.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:02:51.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:02:51.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.733 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10094 3.4 e29/19 3:24263961:::rbd_data.117c6c62fa22.000000000000000b:head v 29'95, mlcod=28'92)", 2024-09-13T05:02:51.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 32.739783795000001, 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001767984, 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:51.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981152+0000", 2024-09-13T05:02:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6986999999999998e-05 2024-09-13T05:02:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981166+0000", 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4331000000000001e-05 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981177+0000", 2024-09-13T05:02:51.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0858e-05 2024-09-13T05:02:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981233+0000", 2024-09-13T05:02:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5813000000000002e-05 2024-09-13T05:02:51.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981253+0000", 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9707999999999998e-05 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982869+0000", 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001615932 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982903+0000", 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4626999999999999e-05 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:19907 3.9 e31/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 31'115, mlcod=31'112)", 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 29.274075045, 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001592883, 2024-09-13T05:02:51.741 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446843+0000", 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446850+0000", 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3450000000000002e-06 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446860+0000", 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446867+0000", 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3570000000000003e-06 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446923+0000", 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6242000000000001e-05 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446940+0000", 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7209000000000001e-05 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448406+0000", 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001465383 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448437+0000", 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1372999999999999e-05 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.745 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.917471149000001, 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:02:51.750 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.776402115, 2024-09-13T05:02:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:02:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:02:51.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:02:51.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.544881465, 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:02:51.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.543905792, 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:02:51.761 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:51.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:02:51.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:02:51.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.543584915, 2024-09-13T05:02:51.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:02:51.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:02:51.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.541496958, 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:02:51.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:02:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:02:51.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:02:51.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:02:51.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:02:51.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:02:51.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:02:51.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:02:51.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:02:51.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.778 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.778 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.778 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:02:51.779 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:51.779 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.447187783, 2024-09-13T05:02:51.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:02:51.779 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.779 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.780 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:51.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:51.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:02:51.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:02:51.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:02:51.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:02:51.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:02:51.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:02:51.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:02:51.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:02:51.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:02:51.787 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:51.788 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.142049322, 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:02:51.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:02:51.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:02:51.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:02:51.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:02:51.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:02:51.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:02:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:02:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:02:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52876 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'388, mlcod=44'385)", 2024-09-13T05:02:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.38273592, 2024-09-13T05:02:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922389, 2024-09-13T05:02:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338182+0000", 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338190+0000", 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.345e-06 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338204+0000", 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4399999999999999e-05 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338212+0000", 2024-09-13T05:02:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3880000000000002e-06 2024-09-13T05:02:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338273+0000", 2024-09-13T05:02:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1030999999999999e-05 2024-09-13T05:02:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338293+0000", 2024-09-13T05:02:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472e-05 2024-09-13T05:02:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340073+0000", 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017803719999999999 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340106+0000", 2024-09-13T05:02:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2186e-05 2024-09-13T05:02:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52877 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'389, mlcod=44'385)", 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.381709605999999, 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00170028, 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339209+0000", 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339215+0000", 2024-09-13T05:02:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5540000000000003e-06 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339226+0000", 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.112e-05 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339233+0000", 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7800000000000003e-06 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339568+0000", 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033527999999999997 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339596+0000", 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7784e-05 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340876+0000", 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012799409999999999 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340910+0000", 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3596000000000002e-05 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52896 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'390, mlcod=44'388)", 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.342635+0000", 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.378284063000001, 2024-09-13T05:02:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016788129999999999, 2024-09-13T05:02:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342635+0000", 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342635+0000", 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342634+0000", 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342644+0000", 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9310000000000003e-06 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342664+0000", 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0463e-05 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342673+0000", 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9449999999999999e-06 2024-09-13T05:02:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342719+0000", 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.604e-05 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342762+0000", 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2728000000000002e-05 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344276+0000", 2024-09-13T05:02:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001514512 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344314+0000", 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7530000000000002e-05 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52897 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'391, mlcod=44'388)", 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.377333588999999, 2024-09-13T05:02:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001898648, 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343585+0000", 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343592+0000", 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9110000000000004e-06 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343602+0000", 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343609+0000", 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9099999999999999e-06 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343673+0000", 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4048999999999996e-05 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343692+0000", 2024-09-13T05:02:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8946000000000001e-05 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345449+0000", 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017568970000000001 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345484+0000", 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5033999999999997e-05 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52899 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'392, mlcod=44'388)", 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.344739+0000", 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.376179862000001, 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00168549, 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344739+0000", 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344739+0000", 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344738+0000", 2024-09-13T05:02:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344745+0000", 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6440000000000003e-06 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344757+0000", 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1799e-05 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344764+0000", 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3209999999999999e-06 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344835+0000", 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1037000000000004e-05 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344857+0000", 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1651e-05 2024-09-13T05:02:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.346391+0000", 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015342649999999999 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.346425+0000", 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3562000000000002e-05 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.362935247999999, 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:02:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:02:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.361648246, 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:02:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:02:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52997 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'399, mlcod=44'395)", 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.360452052999999, 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001838309, 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360466+0000", 2024-09-13T05:02:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360473+0000", 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6150000000000002e-06 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360484+0000", 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1549000000000001e-05 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360495+0000", 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361021+0000", 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000526035 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361040+0000", 2024-09-13T05:02:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8482e-05 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362269+0000", 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012290960000000001 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362305+0000", 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6733000000000002e-05 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:53113 3.4 e44/19 3:2666468c:::rbd_data.117c12c41b07.0000000000000032:head v 44'404, mlcod=44'402)", 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 11.344620256000001, 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017669179999999999, 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376298+0000", 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376382+0000", 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3770999999999995e-05 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376397+0000", 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5688e-05 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376406+0000", 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3459999999999994e-06 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376476+0000", 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0240999999999994e-05 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376502+0000", 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5723000000000001e-05 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.378025+0000", 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001523169 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.378066+0000", 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1072999999999998e-05 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:51.849 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:51.849 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:51.851 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:51.851 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:51.849+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:51.950 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:51.950+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:52.002 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:52.002 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:52.002 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:52.002 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:52.012 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:52.022 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:52.023 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:52.028 INFO:teuthology.orchestra.run.smithi053.stderr:set 3.f pg_upmap_items mapping to [3->5,7->2] 2024-09-13T05:02:52.040 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:52.040 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:52.051 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:52.051+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:52.059 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:52.060 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:52.077 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:52.077 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:52.094 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:52.094 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:52.110 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:52.111 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:52.128 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:52.128 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:52.136 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.136 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.144 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:52.144 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:52.147 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:52.152 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:52.152+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:52.253 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:52.253+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:52.301 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.302 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.310 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:52.355 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:52.354+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:52.412 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.412 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.420 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:52.456 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:52.455+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:52.521 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.521 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.530 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:52.574 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:52.573+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:52.631 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.631 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:52.658 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:52.657+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:52.759 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:52.758+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:52.781 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.782 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.791 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:52.860 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:52.859+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:52.892 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:52.893 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:52.901 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:52.961 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:52.960+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:53.003 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.004 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.012 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:53.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:02:53.098 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:53.097+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:53.155 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.156 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.162 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:53.161+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:53.164 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:53.262 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:53.262+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.315 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:53.363 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:53.363+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:53.456 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.456 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.465 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:53.465 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:53.464+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:53.565 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:53.565+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:53.605 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.606 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.614 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:53.667 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:53.666+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:53.716 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.716 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:53.768 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:53.767+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:53.866 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.866 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.868 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:53.868+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:53.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:53.970 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:53.969+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:53.989 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:53.990 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:53.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:54.036 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:02:54.094 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:54.094+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:54.099 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone27 from 0xdfe37d3 (overlap 0x35507b6) to 0x2551eff 2024-09-13T05:02:54.156 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:54.176 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:54.176 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:54.176 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:54.176+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:54.189 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:54.273 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:54.273+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:54.297 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:54.298 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:54.315 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:54.315 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:54.316 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-13T05:02:54.332 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:54.332 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:54.353 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:54.353 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:54.360 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.703106958999999, 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:02:54.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.703090962000001, 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.703074166, 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.703057192999999, 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:02:54.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.703040799, 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:02:54.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:02:54.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:54.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.703023763000001, 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:02:54.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.70299018, 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:02:54.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.389 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702971050999999, 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:54.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.393 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702893182, 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.397 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702832437000001, 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:02:54.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702772223, 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:02:54.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702754895999998, 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:02:54.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.409 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702725724, 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.410 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:02:54.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702706039999999, 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:02:54.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:02:54.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.417 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702688606999999, 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:02:54.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:02:54.421 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702670964999999, 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.422 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:02:54.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:02:54.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702636789, 2024-09-13T05:02:54.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:02:54.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:54.430 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702611439000002, 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:02:54.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:54.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702593766, 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:54.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:02:54.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.702569562000001, 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:02:54.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.442 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.443 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.443 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:54.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:54.445 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:54.445 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:54.445 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:54.445 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:54.446 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:54.446 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:54.446 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:54.447 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:54.448 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:54.443+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:54.534 INFO:teuthology.orchestra.run.smithi053.stdout:4259 read 0x72f9546 thru 0x73004b1 (0x6f6c bytes) 2024-09-13T05:02:54.536 INFO:teuthology.orchestra.run.smithi053.stdout:4260 writesame 0x2af83a0 thru 0x2afed0b data_size 0x4e(0x696c bytes) 2024-09-13T05:02:54.545 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:54.544+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:54.552 INFO:teuthology.orchestra.run.smithi053.stdout:4263 read 0x2a47d59 thru 0x2a49e6f (0x2117 bytes) 2024-09-13T05:02:54.566 INFO:teuthology.orchestra.run.smithi053.stdout:4265 trunc from 0xdfe37d3 to 0x1083eae 2024-09-13T05:02:54.619 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:54.619 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:54.619 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:54.619 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:54.630 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:54.671 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:54.671+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:54.681 INFO:teuthology.orchestra.run.smithi053.stdout:4266 read 0x2c4f thru 0x11e4c (0xf1fe bytes) 2024-09-13T05:02:54.690 INFO:teuthology.orchestra.run.smithi053.stdout:4267 writesame 0x45834ae thru 0x4590e5d data_size 0x2b(0xd9b0 bytes) 2024-09-13T05:02:54.690 INFO:teuthology.orchestra.run.smithi053.stdout:4268 read 0x421ac88 thru 0x4229c63 (0xefdc bytes) 2024-09-13T05:02:54.690 INFO:teuthology.orchestra.run.smithi053.stdout:4269 punch from 0x1bd331 to 0x1c0c18, (0x38e7 bytes) 2024-09-13T05:02:54.693 INFO:teuthology.orchestra.run.smithi053.stdout:4270 trunc from 0x4590e5e to 0x23e8f00 2024-09-13T05:02:54.713 INFO:teuthology.orchestra.run.smithi053.stdout:4271 write 0xa947054 thru 0xa948a87 (0x1a34 bytes) 2024-09-13T05:02:54.716 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:54.716 INFO:teuthology.orchestra.run.smithi053.stdout:4274 punch from 0x20941e7 to 0x209e2f5, (0xa10e bytes) 2024-09-13T05:02:54.724 INFO:teuthology.orchestra.run.smithi053.stdout:4276 write 0xd52fac6 thru 0xd53e83d (0xed78 bytes) 2024-09-13T05:02:54.726 INFO:teuthology.orchestra.run.smithi053.stdout:4277 punch from 0x24ed73c to 0x24f9189, (0xba4d bytes) 2024-09-13T05:02:54.726 INFO:teuthology.orchestra.run.smithi053.stdout:4278 read 0xe81f6d thru 0xe82047 (0xdb bytes) 2024-09-13T05:02:54.727 INFO:teuthology.orchestra.run.smithi053.stdout:4283 writesame 0x2a5bec5 thru 0x2a6ba8a data_size 0x25(0xfbc6 bytes) 2024-09-13T05:02:54.730 INFO:teuthology.orchestra.run.smithi053.stdout:4285 read 0x1b612f0 thru 0x1b67bfd (0x690e bytes) 2024-09-13T05:02:54.740 INFO:teuthology.orchestra.run.smithi053.stdout:4287 trunc from 0xd53e83e to 0x4face24 2024-09-13T05:02:54.747 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:54.746+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:54.756 INFO:teuthology.orchestra.run.smithi053.stdout:4288 write 0x6b31ded thru 0x6b3b9e8 (0x9bfc bytes) 2024-09-13T05:02:54.766 INFO:teuthology.orchestra.run.smithi053.stdout:4289 trunc from 0x6b3b9e9 to 0xbcc001 2024-09-13T05:02:54.788 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:54.788 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:54.789 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:54.789 INFO:teuthology.orchestra.run.smithi053.stdout:4290 write 0x1dd8644 thru 0x1de23c8 (0x9d85 bytes) 2024-09-13T05:02:54.791 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:54.791 INFO:teuthology.orchestra.run.smithi053.stdout:4294 punch from 0xd1ec25 to 0xd264b2, (0x788d bytes) 2024-09-13T05:02:54.799 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:54.800 INFO:teuthology.orchestra.run.smithi053.stdout:4295 write 0x4874442 thru 0x487f4a5 (0xb064 bytes) 2024-09-13T05:02:54.802 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:54.811 INFO:teuthology.orchestra.run.smithi053.stdout:4298 writesame 0x6e23201 thru 0x6e322da data_size 0x1(0xf0da bytes) 2024-09-13T05:02:54.811 INFO:teuthology.orchestra.run.smithi053.stdout:4299 read 0x63744d5 thru 0x63824ee (0xe01a bytes) 2024-09-13T05:02:54.817 INFO:teuthology.orchestra.run.smithi053.stdout:4300 writesame 0x91b9808 thru 0x91c149d data_size 0x25(0x7c96 bytes) 2024-09-13T05:02:54.828 INFO:teuthology.orchestra.run.smithi053.stdout:4301 writesame 0xd7aac7f thru 0xd7ba721 data_size 0x13(0xfaa3 bytes) 2024-09-13T05:02:54.828 INFO:teuthology.orchestra.run.smithi053.stdout:4302 write 0x63b8e7 thru 0x6474a1 (0xbbbb bytes) 2024-09-13T05:02:54.830 INFO:teuthology.orchestra.run.smithi053.stdout:4305 write 0x4a05590 thru 0x4a09fa3 (0x4a14 bytes) 2024-09-13T05:02:54.833 INFO:teuthology.orchestra.run.smithi053.stdout:4306 write 0x505cbad thru 0x505e730 (0x1b84 bytes) 2024-09-13T05:02:54.836 INFO:teuthology.orchestra.run.smithi053.stdout:4308 punch from 0x43875e7 to 0x438b8f7, (0x4310 bytes) 2024-09-13T05:02:54.836 INFO:teuthology.orchestra.run.smithi053.stdout:4309 read 0x62e67b8 thru 0x62f0ec8 (0xa711 bytes) 2024-09-13T05:02:54.836 INFO:teuthology.orchestra.run.smithi053.stdout:4310 read 0xa2a072c thru 0xa2aacf4 (0xa5c9 bytes) 2024-09-13T05:02:54.836 INFO:teuthology.orchestra.run.smithi053.stdout:4311 write 0xd328c0d thru 0xd336254 (0xd648 bytes) 2024-09-13T05:02:54.839 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:54.848 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:54.847+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:54.854 INFO:teuthology.orchestra.run.smithi053.stdout:4313 trunc from 0xd7ba722 to 0xc15501 2024-09-13T05:02:54.879 INFO:teuthology.orchestra.run.smithi053.stdout:4314 writesame 0x8144bf0 thru 0x8144ee3 data_size 0xc(0x2f4 bytes) 2024-09-13T05:02:54.882 INFO:teuthology.orchestra.run.smithi053.stdout:4316 trunc from 0x8144ee4 to 0x5ab2b49 2024-09-13T05:02:54.888 INFO:teuthology.orchestra.run.smithi053.stdout:4318 punch from 0x30cca1d to 0x30d747b, (0xaa5e bytes) 2024-09-13T05:02:54.888 INFO:teuthology.orchestra.run.smithi053.stdout:4319 read 0xe9ab54 thru 0xea09f2 (0x5e9f bytes) 2024-09-13T05:02:54.889 INFO:teuthology.orchestra.run.smithi053.stdout:4320 punch from 0xcbdf65 to 0xccc9cd, (0xea68 bytes) 2024-09-13T05:02:54.889 INFO:teuthology.orchestra.run.smithi053.stdout:4321 punch from 0x406ca4 to 0x412873, (0xbbcf bytes) 2024-09-13T05:02:54.894 INFO:teuthology.orchestra.run.smithi053.stdout:4322 write 0x72b7ebb thru 0x72c320c (0xb352 bytes) 2024-09-13T05:02:54.897 INFO:teuthology.orchestra.run.smithi053.stdout:4324 read 0xcfe906 thru 0xd07ebd (0x95b8 bytes) 2024-09-13T05:02:54.905 INFO:teuthology.orchestra.run.smithi053.stdout:4325 write 0x95a7412 thru 0x95a9081 (0x1c70 bytes) 2024-09-13T05:02:54.908 INFO:teuthology.orchestra.run.smithi053.stdout:4326 read 0x5b1a9cf thru 0x5b1db0d (0x313f bytes) 2024-09-13T05:02:54.914 INFO:teuthology.orchestra.run.smithi053.stdout:4327 trunc from 0x95a9082 to 0x4bfe8e5 2024-09-13T05:02:54.923 INFO:teuthology.orchestra.run.smithi053.stdout:4329 read 0x41bebf3 thru 0x41cd16a (0xe578 bytes) 2024-09-13T05:02:54.923 INFO:teuthology.orchestra.run.smithi053.stdout:4330 read 0x143f01d thru 0x144e2cc (0xf2b0 bytes) 2024-09-13T05:02:54.923 INFO:teuthology.orchestra.run.smithi053.stdout:4332 read 0x2a9b125 thru 0x2aa3d6e (0x8c4a bytes) 2024-09-13T05:02:54.923 INFO:teuthology.orchestra.run.smithi053.stdout:4333 read 0x3b99058 thru 0x3ba871b (0xf6c4 bytes) 2024-09-13T05:02:54.931 INFO:teuthology.orchestra.run.smithi053.stdout:4334 writesame 0x693e8e3 thru 0x694d642 data_size 0x60(0xed60 bytes) 2024-09-13T05:02:54.931 INFO:teuthology.orchestra.run.smithi053.stdout:4335 read 0x26252c3 thru 0x262a70e (0x544c bytes) 2024-09-13T05:02:54.931 INFO:teuthology.orchestra.run.smithi053.stdout:4337 punch from 0x4b122e to 0x4bd5d3, (0xc3a5 bytes) 2024-09-13T05:02:54.931 INFO:teuthology.orchestra.run.smithi053.stdout:4338 read 0x4673fef thru 0x467534e (0x1360 bytes) 2024-09-13T05:02:54.931 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:54.931 INFO:teuthology.orchestra.run.smithi053.stdout:4341 write 0x6908080 thru 0x6915469 (0xd3ea bytes) 2024-09-13T05:02:54.932 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568956612000001, 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.933 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:02:54.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:02:54.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:54.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568933568999999, 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:54.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568916237, 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:02:54.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:02:54.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568899692999999, 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:54.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:02:54.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568882567999999, 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:02:54.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.56886634, 2024-09-13T05:02:54.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:02:54.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568849529000001, 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:02:54.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:02:54.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568832911000001, 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.970 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:02:54.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568816333000001, 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.975 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:02:54.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:54.980 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568779978999999, 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:02:54.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:02:54.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568761035000001, 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:02:54.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:02:54.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568744401, 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:54.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:02:54.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:54.996 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568718400000002, 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:54.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:54.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568686460999999, 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:02:55.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:02:55.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568667674, 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:02:55.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568651553999999, 2024-09-13T05:02:55.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:02:55.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.017 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568629493, 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.568612527999999, 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.023 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:55.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:02:55.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.027 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.256578129000001, 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:02:55.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.032 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.256559021000001, 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:55.033 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:02:55.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:02:55.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.038 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:55.039 INFO:teuthology.orchestra.run.smithi053.stdout:4342 write 0x94f7daf thru 0x9501a30 (0x9c82 bytes) 2024-09-13T05:02:55.039 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:55.039+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:55.041 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:55.042 INFO:teuthology.orchestra.run.smithi053.stdout:4343 writesame 0xc4d5beb thru 0xc4df02f data_size 0x1(0x9445 bytes) 2024-09-13T05:02:55.042 INFO:teuthology.orchestra.run.smithi053.stdout:4344 punch from 0x3dc0475 to 0x3dc15e4, (0x116f bytes) 2024-09-13T05:02:55.042 INFO:teuthology.orchestra.run.smithi053.stdout:4345 write 0x8ef1f91 thru 0x8effb86 (0xdbf6 bytes) 2024-09-13T05:02:55.042 INFO:teuthology.orchestra.run.smithi053.stdout:4346 write 0x63a3811 thru 0x63a6979 (0x3169 bytes) 2024-09-13T05:02:55.042 INFO:teuthology.orchestra.run.smithi053.stdout:4348 punch from 0x51d43a7 to 0x51d5912, (0x156b bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4349 writesame 0x3602be1 thru 0x36079c4 data_size 0xa(0x4de4 bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4351 punch from 0x5802b4a to 0x5811f6f, (0xf425 bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4352 read 0x616a0dd thru 0x61788a2 (0xe7c6 bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4353 write 0x4a0e257 thru 0x4a133bb (0x5165 bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4354 read 0x2f01ca7 thru 0x2f0966e (0x79c8 bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4355 punch from 0x1fdc5a3 to 0x1feaacd, (0xe52a bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4356 write 0x428b560 thru 0x4299a3b (0xe4dc bytes) 2024-09-13T05:02:55.043 INFO:teuthology.orchestra.run.smithi053.stdout:4357 clone 29 order 18 su 8192 sc 15 2024-09-13T05:02:55.140 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:55.140+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:55.189 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:55.189 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:55.189 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:55.189 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:55.198 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:55.241 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:55.240+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:55.342 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:55.341+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:55.345 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:55.345 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:55.345 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:55.345 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:55.345 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:55.354 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:55.442 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:55.442+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:55.502 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.523317300000002, 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.505 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:02:55.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10094 3.4 e29/19 3:24263961:::rbd_data.117c6c62fa22.000000000000000b:head v 29'95, mlcod=28'92)", 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.520245891000002, 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001767984, 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981152+0000", 2024-09-13T05:02:55.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6986999999999998e-05 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981166+0000", 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4331000000000001e-05 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981177+0000", 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0858e-05 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981233+0000", 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5813000000000002e-05 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981253+0000", 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9707999999999998e-05 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982869+0000", 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001615932 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982903+0000", 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4626999999999999e-05 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:19907 3.9 e31/19 3:93357c4c:::rbd_object_map.117c3521aa89:head v 31'115, mlcod=31'112)", 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 33.054537140999997, 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001592883, 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.512 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446844+0000", 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446843+0000", 2024-09-13T05:02:55.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446850+0000", 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3450000000000002e-06 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446860+0000", 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446867+0000", 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3570000000000003e-06 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446923+0000", 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6242000000000001e-05 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.446940+0000", 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7209000000000001e-05 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448406+0000", 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001465383 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:22.448437+0000", 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1372999999999999e-05 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.516 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.697933245000002, 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.556864211000001, 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.521 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:02:55.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:02:55.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:55.525 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.325343561, 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:55.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:02:55.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:02:55.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:02:55.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.324367888000001, 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:02:55.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:02:55.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:02:55.535 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.324047011000001, 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:55.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:02:55.539 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.321959054000001, 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.541 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:02:55.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:02:55.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:55.545 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.227649879000001, 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:02:55.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:02:55.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.922511417999999, 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.550 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:02:55.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52876 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'388, mlcod=44'385)", 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.163198016000001, 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922389, 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338182+0000", 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338190+0000", 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.345e-06 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338204+0000", 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4399999999999999e-05 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338212+0000", 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3880000000000002e-06 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338273+0000", 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1030999999999999e-05 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338293+0000", 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472e-05 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340073+0000", 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017803719999999999 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340106+0000", 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2186e-05 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52877 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'389, mlcod=44'385)", 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.162171702, 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00170028, 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.559 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339209+0000", 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339215+0000", 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5540000000000003e-06 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339226+0000", 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.112e-05 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339233+0000", 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7800000000000003e-06 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339568+0000", 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033527999999999997 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339596+0000", 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7784e-05 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340876+0000", 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012799409999999999 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340910+0000", 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3596000000000002e-05 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52896 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'390, mlcod=44'388)", 2024-09-13T05:02:55.563 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.342635+0000", 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.158746159, 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016788129999999999, 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342635+0000", 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342635+0000", 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342634+0000", 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342644+0000", 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9310000000000003e-06 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342664+0000", 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0463e-05 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342673+0000", 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9449999999999999e-06 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342719+0000", 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.604e-05 2024-09-13T05:02:55.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.342762+0000", 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2728000000000002e-05 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344276+0000", 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001514512 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344314+0000", 2024-09-13T05:02:55.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7530000000000002e-05 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52897 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'391, mlcod=44'388)", 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.157795685, 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001898648, 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.568 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343585+0000", 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343592+0000", 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9110000000000004e-06 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343602+0000", 2024-09-13T05:02:55.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343609+0000", 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9099999999999999e-06 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343673+0000", 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4048999999999996e-05 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343692+0000", 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8946000000000001e-05 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345449+0000", 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017568970000000001 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345484+0000", 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5033999999999997e-05 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.572 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52899 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'392, mlcod=44'388)", 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.344739+0000", 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.156641958, 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00168549, 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344739+0000", 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344739+0000", 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344738+0000", 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344745+0000", 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6440000000000003e-06 2024-09-13T05:02:55.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344757+0000", 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1799e-05 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344764+0000", 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3209999999999999e-06 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344835+0000", 2024-09-13T05:02:55.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1037000000000004e-05 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.344857+0000", 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1651e-05 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.346391+0000", 2024-09-13T05:02:55.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015342649999999999 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.346425+0000", 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3562000000000002e-05 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:55.577 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.143397344, 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:02:55.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.581 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.142110342000001, 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:55.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:02:55.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52997 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'399, mlcod=44'395)", 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.140914149, 2024-09-13T05:02:55.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001838309, 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360466+0000", 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360473+0000", 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6150000000000002e-06 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360484+0000", 2024-09-13T05:02:55.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1549000000000001e-05 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360495+0000", 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361021+0000", 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000526035 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361040+0000", 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8482e-05 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362269+0000", 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012290960000000001 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362305+0000", 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6733000000000002e-05 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:53113 3.4 e44/19 3:2666468c:::rbd_data.117c12c41b07.0000000000000032:head v 44'404, mlcod=44'402)", 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.125082352, 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017669179999999999, 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376298+0000", 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376382+0000", 2024-09-13T05:02:55.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3770999999999995e-05 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376397+0000", 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5688e-05 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376406+0000", 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3459999999999994e-06 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:55.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376476+0000", 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0240999999999994e-05 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376502+0000", 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5723000000000001e-05 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.378025+0000", 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001523169 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:55.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.378066+0000", 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1072999999999998e-05 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:55.595 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:55.596 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:55.597 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:55.595+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:55.698 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:55.697+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:55.704 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:55.705 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:55.705 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:55.705 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:55.713 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:55.799 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:55.798+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:55.859 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:55.860 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:55.860 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:55.860 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:55.860 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:55.868 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:55.911 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:55.910+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:55.993 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone28 intact 2024-09-13T05:02:56.000 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:56.000+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.50755659, 2024-09-13T05:02:56.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:02:56.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.507385284999998, 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:02:56.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:02:56.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47845 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1564672~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.365919+0000", 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.659532036000002, 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017669296000000001, 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.039 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47845 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365919+0000", 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365919+0000", 2024-09-13T05:02:56.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365918+0000", 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365921+0000", 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0000000000000001e-06 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365929+0000", 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.678e-06 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.365935+0000", 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383394+0000", 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017459005999999999 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383485+0000", 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1011e-05 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383588+0000", 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010336900000000001 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.043 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48086 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 696320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.657067852000001, 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017753149999999999, 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48086 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368383+0000", 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368384+0000", 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.829e-06 2024-09-13T05:02:56.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368388+0000", 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000002e-06 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368392+0000", 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5300000000000001e-06 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385946+0000", 2024-09-13T05:02:56.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017554422 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386034+0000", 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7174000000000003e-05 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386136+0000", 2024-09-13T05:02:56.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010249799999999999 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48087 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 753664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368569+0000", 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656881722000001, 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017776863, 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48087 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368569+0000", 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368569+0000", 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368568+0000", 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368573+0000", 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3259999999999997e-06 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368584+0000", 2024-09-13T05:02:56.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1394e-05 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368589+0000", 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9219999999999996e-06 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386166+0000", 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017577354 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386250+0000", 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4167999999999998e-05 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386346+0000", 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5478999999999998e-05 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48088 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 794624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368601+0000", 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656850123000002, 2024-09-13T05:02:56.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017953779999999999, 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48088 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368601+0000", 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368601+0000", 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368600+0000", 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368602+0000", 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2000000000000001e-06 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368607+0000", 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2100000000000003e-06 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368609+0000", 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.841e-06 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386388+0000", 2024-09-13T05:02:56.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017778828999999999 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386451+0000", 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3146999999999997e-05 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386554+0000", 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000102942 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48089 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 851968~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368690+0000", 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656760773999999, 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018046405000000001, 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48089 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368690+0000", 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368690+0000", 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368689+0000", 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368694+0000", 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0810000000000004e-06 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368702+0000", 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4339999999999992e-06 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368708+0000", 2024-09-13T05:02:56.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7359999999999998e-06 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386596+0000", 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017887888000000001 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386680+0000", 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4277000000000006e-05 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386736+0000", 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6523000000000001e-05 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48090 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 909312~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368820+0000", 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656630440000001, 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018075834999999998, 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48090 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368820+0000", 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368820+0000", 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368819+0000", 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368824+0000", 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8060000000000002e-06 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368835+0000", 2024-09-13T05:02:56.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0403e-05 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368843+0000", 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1210000000000007e-06 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386766+0000", 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017923005999999998 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386820+0000", 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4191999999999999e-05 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386896+0000", 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6147000000000001e-05 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48091 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 950272~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.368896+0000", 2024-09-13T05:02:56.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656554562, 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018221838000000001, 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48091 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368896+0000", 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368896+0000", 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368895+0000", 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368899+0000", 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0119999999999999e-06 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368908+0000", 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8319999999999995e-06 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.368914+0000", 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9290000000000003e-06 2024-09-13T05:02:56.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.386937+0000", 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.01802287 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387026+0000", 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9347999999999999e-05 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387118+0000", 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1694000000000007e-05 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48092 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656390321, 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018282850999999999, 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48092 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369060+0000", 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369064+0000", 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6589999999999998e-06 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369074+0000", 2024-09-13T05:02:56.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4539999999999996e-06 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369080+0000", 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6320000000000002e-06 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387156+0000", 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018075872999999999 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387246+0000", 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9963999999999994e-05 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387343+0000", 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7156000000000003e-05 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48093 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1048576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:56.075 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656190341999999, 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018225456000000001, 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48093 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369260+0000", 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369264+0000", 2024-09-13T05:02:56.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6659999999999999e-06 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369274+0000", 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4970000000000007e-06 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369280+0000", 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4400000000000002e-06 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387381+0000", 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018101092999999999 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387433+0000", 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1823000000000002e-05 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387486+0000", 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2704000000000002e-05 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.079 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48094 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1105920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.656020378000001, 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018210301000000002, 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48094 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.080 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369430+0000", 2024-09-13T05:02:56.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369434+0000", 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5349999999999998e-06 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369446+0000", 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1445e-05 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369466+0000", 2024-09-13T05:02:56.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0044999999999999e-05 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387524+0000", 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018058576 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387580+0000", 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5550999999999997e-05 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387641+0000", 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0965999999999999e-05 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48095 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1163264~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655789240000001, 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018110261999999999, 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.084 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48095 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:56.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369661+0000", 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369666+0000", 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3669999999999996e-06 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369677+0000", 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0873000000000001e-05 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369685+0000", 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1689999999999992e-06 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387679+0000", 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017993776999999999 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387720+0000", 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0929000000000001e-05 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387772+0000", 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2108000000000003e-05 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.088 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48096 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1204224~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.369909+0000", 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655541660000001, 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018114216999999998, 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48096 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369909+0000", 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369909+0000", 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369908+0000", 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369913+0000", 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4970000000000003e-06 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369922+0000", 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2399999999999996e-06 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.369928+0000", 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4979999999999997e-06 2024-09-13T05:02:56.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387845+0000", 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017917807000000001 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.387930+0000", 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4864999999999995e-05 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388023+0000", 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3066000000000006e-05 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48097 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1261568~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370061+0000", 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655389547999999, 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018148152000000001, 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.093 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48097 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370061+0000", 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370061+0000", 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370060+0000", 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370066+0000", 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9869999999999996e-06 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370077+0000", 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0844e-05 2024-09-13T05:02:56.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370084+0000", 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5150000000000003e-06 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388066+0000", 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.017981967000000001 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388149+0000", 2024-09-13T05:02:56.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3150000000000002e-05 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388209+0000", 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0448999999999997e-05 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48098 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1359872~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370102+0000", 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655349147999999, 2024-09-13T05:02:56.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018247026999999999, 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48098 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370102+0000", 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370102+0000", 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370101+0000", 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370105+0000", 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8469999999999998e-06 2024-09-13T05:02:56.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370112+0000", 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6229999999999997e-06 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370118+0000", 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0610000000000001e-06 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388238+0000", 2024-09-13T05:02:56.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018119988 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388291+0000", 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2264e-05 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388349+0000", 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7865999999999998e-05 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48099 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1417216~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370124+0000", 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.65532683, 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018485003999999999, 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48099 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370124+0000", 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370124+0000", 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370123+0000", 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370126+0000", 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.61e-06 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370133+0000", 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6869999999999997e-06 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370138+0000", 2024-09-13T05:02:56.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0420000000000002e-06 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388400+0000", 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018262332999999999 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388501+0000", 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010084 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388609+0000", 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010792400000000001 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48100 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655307291, 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018661594, 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48100 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370143+0000", 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370145+0000", 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370151+0000", 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0220000000000001e-06 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370156+0000", 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9729999999999997e-06 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388650+0000", 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018493451000000001 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388704+0000", 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4401000000000003e-05 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388805+0000", 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010076 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:48101 3.8 3:14ef1248:::rbd_data.117c6173c20e.0000000000000007:14 [read 1515520~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:56.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 18.655286774, 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018784986, 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 48101 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:56.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370164+0000", 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370166+0000", 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370173+0000", 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6000000000000003e-06 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.370177+0000", 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9869999999999999e-06 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388844+0000", 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018666312000000001 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388893+0000", 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9604000000000001e-05 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.388949+0000", 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5529000000000003e-05 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.114 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:54861 3.1e 3:7a435ed0:::rbd_data.117c12c41b07.0000000000000067:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44)", 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:41.233984+0000", 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.7914671, 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018839166000000001, 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 54861 2024-09-13T05:02:56.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.233984+0000", 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.233984+0000", 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:56.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.233982+0000", 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.233991+0000", 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4890000000000001e-06 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.234009+0000", 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7331000000000001e-05 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.234018+0000", 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2669999999999995e-06 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.234067+0000", 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9101000000000003e-05 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.234156+0000", 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8675000000000001e-05 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 6", 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.237443+0000", 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003287145 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.252689+0000", 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015246103 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.252703+0000", 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4414e-05 2024-09-13T05:02:56.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.252761+0000", 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8090000000000001e-05 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:41.252823+0000", 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1452999999999996e-05 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:56.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:56.121 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:56.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:02:56.122 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:02:56.123 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:56.121+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:56.219 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-13T05:02:56.223 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:56.222+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:56.241 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:56.241 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:56.241 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:02:56.241 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:56.249 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:02:56.334 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:56.333+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:56.391 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:56.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:02:56.391 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:02:56.392 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:02:56.392 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:56.400 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:02:56.442 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:56.441+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:56.448 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:02:56.510 INFO:teuthology.orchestra.run.smithi053.stdout:4358 punch from 0x96f6e89 to 0x970690c, (0xfa83 bytes) 2024-09-13T05:02:56.514 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.515 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.515 INFO:teuthology.orchestra.run.smithi053.stdout:4363 read 0xb1e6618 thru 0xb1f188d (0xb276 bytes) 2024-09-13T05:02:56.515 INFO:teuthology.orchestra.run.smithi053.stdout:4364 write 0xb17e677 thru 0xb1877b5 (0x913f bytes) 2024-09-13T05:02:56.522 INFO:teuthology.orchestra.run.smithi053.stdout:4365 read 0x94b4505 thru 0x94bdfaa (0x9aa6 bytes) 2024-09-13T05:02:56.526 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:56.526+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:56.535 INFO:teuthology.orchestra.run.smithi053.stdout:4366 trunc from 0xc4df030 to 0x19b1103 2024-09-13T05:02:56.549 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:02:56.549 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305307604999999, 2024-09-13T05:02:56.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:56.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:02:56.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305291350000001, 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:02:56.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:02:56.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.560 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305274261000001, 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305257394000002, 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.565 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.570 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305208096000001, 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:02:56.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305184827000001, 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:02:56.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:02:56.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:02:56.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:56.581 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305169093, 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:56.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:02:56.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305146112999999, 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:56.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:02:56.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.305097047, 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:02:56.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:02:56.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304975731999999, 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.598 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304920517999999, 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:02:56.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:02:56.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:02:56.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304900229000001, 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.608 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:02:56.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:02:56.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:02:56.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304883048000001, 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:02:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:02:56.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304795538, 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:02:56.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:02:56.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:02:56.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.623 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.30477629, 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:02:56.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:02:56.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304715174999998, 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:02:56.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:56.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.634 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304697999999998, 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:02:56.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304681590000001, 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:02:56.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:02:56.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:02:56.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:02:56.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:02:56.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:02:56.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:02:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:02:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304660605999999, 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.646 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:56.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:02:56.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:02:56.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:02:56.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 29.304643182, 2024-09-13T05:02:56.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:02:56.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:02:56.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:02:56.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:02:56.659 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:02:56.660 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:56.661 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:56.661 INFO:teuthology.orchestra.run.smithi053.stdout:4368 write 0xb560085 thru 0xb565661 (0x55dd bytes) 2024-09-13T05:02:56.661 INFO:teuthology.orchestra.run.smithi053.stdout:4369 write 0x5176007 thru 0x518436a (0xe364 bytes) 2024-09-13T05:02:56.662 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:56.663 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:56.663 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:02:56.664 INFO:teuthology.orchestra.run.smithi053.stdout:4370 trunc from 0xb565662 to 0x754f211 2024-09-13T05:02:56.664 INFO:teuthology.orchestra.run.smithi053.stdout:4371 trunc from 0x754f211 to 0x41f2b0b 2024-09-13T05:02:56.664 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:56.665 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:56.666 INFO:teuthology.orchestra.run.smithi053.stdout:4372 punch from 0x8c938b to 0x8ce8ce, (0x5543 bytes) 2024-09-13T05:02:56.666 INFO:teuthology.orchestra.run.smithi053.stdout:4374 read 0x2c2de86 thru 0x2c3ba16 (0xdb91 bytes) 2024-09-13T05:02:56.666 INFO:teuthology.orchestra.run.smithi053.stdout:4376 trunc from 0x41f2b0b to 0x5ed7f0a 2024-09-13T05:02:56.666 INFO:teuthology.orchestra.run.smithi053.stdout:4377 writesame 0x54bf176 thru 0x54ce42a data_size 0x3(0xf2b5 bytes) 2024-09-13T05:02:56.667 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:56.668 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:56.668 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:56.660+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:56.679 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:56.679 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:56.683 INFO:teuthology.orchestra.run.smithi053.stdout:4380 write 0xd0a176f thru 0xd0a1fb6 (0x848 bytes) 2024-09-13T05:02:56.686 INFO:teuthology.orchestra.run.smithi053.stdout:4381 punch from 0x13bdfa2 to 0x13c60f2, (0x8150 bytes) 2024-09-13T05:02:56.686 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.690 INFO:teuthology.orchestra.run.smithi053.stdout:4383 trunc from 0xd0a1fb7 to 0x8e75a90 2024-09-13T05:02:56.696 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:56.696 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:56.700 INFO:teuthology.orchestra.run.smithi053.stdout:4384 read 0x3e13d5 thru 0x3ef4cf (0xe0fb bytes) 2024-09-13T05:02:56.701 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.702 INFO:teuthology.orchestra.run.smithi053.stdout:4388 read 0x2c0f7a5 thru 0x2c14eb4 (0x5710 bytes) 2024-09-13T05:02:56.702 INFO:teuthology.orchestra.run.smithi053.stdout:4390 read 0x8627fb5 thru 0x8633c79 (0xbcc5 bytes) 2024-09-13T05:02:56.707 INFO:teuthology.orchestra.run.smithi053.stdout:4391 trunc from 0x8e75a90 to 0xdd9c801 2024-09-13T05:02:56.713 INFO:teuthology.orchestra.run.smithi053.stdout:4392 read 0x43a2680 thru 0x43abfd6 (0x9957 bytes) 2024-09-13T05:02:56.713 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.713 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.714 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:56.715 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:56.715 INFO:teuthology.orchestra.run.smithi053.stdout:4395 trunc from 0xdd9c801 to 0xc16d932 2024-09-13T05:02:56.720 INFO:teuthology.orchestra.run.smithi053.stdout:4396 read 0x6f3f82d thru 0x6f4bbcb (0xc39f bytes) 2024-09-13T05:02:56.720 INFO:teuthology.orchestra.run.smithi053.stdout:4398 punch from 0xbf68c3c to 0xbf78c30, (0xfff4 bytes) 2024-09-13T05:02:56.720 INFO:teuthology.orchestra.run.smithi053.stdout:4401 punch from 0x54e517f to 0x54eb907, (0x6788 bytes) 2024-09-13T05:02:56.721 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.721 INFO:teuthology.orchestra.run.smithi053.stdout:4406 writesame 0x6594ba5 thru 0x659d57e data_size 0x5(0x89da bytes) 2024-09-13T05:02:56.728 INFO:teuthology.orchestra.run.smithi053.stdout:4408 write 0x371fb4f thru 0x372eca9 (0xf15b bytes) 2024-09-13T05:02:56.734 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:56.734 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:56.739 INFO:teuthology.orchestra.run.smithi053.stdout:4409 punch from 0x3d437cc to 0x3d46731, (0x2f65 bytes) 2024-09-13T05:02:56.739 INFO:teuthology.orchestra.run.smithi053.stdout:4410 writesame 0x50085fe thru 0x50129b1 data_size 0x2(0xa3b4 bytes) 2024-09-13T05:02:56.748 INFO:teuthology.orchestra.run.smithi053.stdout:4411 writesame 0x27b7bfb thru 0x27c045f data_size 0x3(0x8865 bytes) 2024-09-13T05:02:56.757 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.757 INFO:teuthology.orchestra.run.smithi053.stdout:4413 punch from 0x2e215c6 to 0x2e2389c, (0x22d6 bytes) 2024-09-13T05:02:56.757 INFO:teuthology.orchestra.run.smithi053.stdout:4414 write 0x1a3b6c0 thru 0x1a3e931 (0x3272 bytes) 2024-09-13T05:02:56.759 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.759 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.759 INFO:teuthology.orchestra.run.smithi053.stdout:4419 write 0x72f7628 thru 0x72f85c4 (0xf9d bytes) 2024-09-13T05:02:56.761 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:56.760+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:56.765 INFO:teuthology.orchestra.run.smithi053.stdout:4420 trunc from 0xc16d932 to 0x93f16c2 2024-09-13T05:02:56.774 INFO:teuthology.orchestra.run.smithi053.stdout:4421 trunc from 0x93f16c2 to 0x603ecfa 2024-09-13T05:02:56.784 INFO:teuthology.orchestra.run.smithi053.stdout:4422 read 0x45d6c3d thru 0x45e0b3f (0x9f03 bytes) 2024-09-13T05:02:56.785 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:56.785 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:56.789 INFO:teuthology.orchestra.run.smithi053.stdout:4423 trunc from 0x603ecfa to 0xa903af1 2024-09-13T05:02:56.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:02:56.795 INFO:teuthology.orchestra.run.smithi053.stdout:4424 read 0x4d8b422 thru 0x4d97f8c (0xcb6b bytes) 2024-09-13T05:02:56.795 INFO:teuthology.orchestra.run.smithi053.stdout:4425 write 0x15d41b7 thru 0x15e39f2 (0xf83c bytes) 2024-09-13T05:02:56.815 INFO:teuthology.orchestra.run.smithi053.stdout:4426 writesame 0xeb417f0 thru 0xeb4e6c3 data_size 0x1c(0xced4 bytes) 2024-09-13T05:02:56.816 INFO:teuthology.orchestra.run.smithi053.stdout:4427 punch from 0xb0b86ae to 0xb0c2a1b, (0xa36d bytes) 2024-09-13T05:02:56.820 INFO:teuthology.orchestra.run.smithi053.stdout:4428 write 0xede637b thru 0xedebe22 (0x5aa8 bytes) 2024-09-13T05:02:56.825 INFO:teuthology.orchestra.run.smithi053.stdout:4430 writesame 0xbe9903d thru 0xbea4cab data_size 0x1(0xbc6f bytes) 2024-09-13T05:02:56.837 INFO:teuthology.orchestra.run.smithi053.stdout:4432 read 0x295b02b thru 0x2968950 (0xd926 bytes) 2024-09-13T05:02:56.840 INFO:teuthology.orchestra.run.smithi053.stdout:4435 trunc from 0xedebe23 to 0xc91425f 2024-09-13T05:02:56.850 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.850 INFO:teuthology.orchestra.run.smithi053.stdout:4437 write 0x5cae44c thru 0x5cb1da3 (0x3958 bytes) 2024-09-13T05:02:56.859 INFO:teuthology.orchestra.run.smithi053.stdout:4438 write 0xdf36cc9 thru 0xdf3d5e3 (0x691b bytes) 2024-09-13T05:02:56.862 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:56.861+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:56.865 INFO:teuthology.orchestra.run.smithi053.stdout:4440 punch from 0x67cf0e9 to 0x67dbc6b, (0xcb82 bytes) 2024-09-13T05:02:56.866 INFO:teuthology.orchestra.run.smithi053.stdout:4441 read 0xc7f4e76 thru 0xc7fda64 (0x8bef bytes) 2024-09-13T05:02:56.881 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:02:56.881 INFO:teuthology.orchestra.run.smithi053.stdout:4442 trunc from 0xdf3d5e4 to 0x1826f32 2024-09-13T05:02:56.903 INFO:teuthology.orchestra.run.smithi053.stdout:4443 read 0x13c49b8 thru 0x13d0d76 (0xc3bf bytes) 2024-09-13T05:02:56.914 INFO:teuthology.orchestra.run.smithi053.stdout:4444 trunc from 0x1826f32 to 0xac1db55 2024-09-13T05:02:56.918 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:56.918 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:56.920 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:56.920 INFO:teuthology.orchestra.run.smithi053.stdout:4446 read 0x76a7839 thru 0x76b4a1e (0xd1e6 bytes) 2024-09-13T05:02:56.920 INFO:teuthology.orchestra.run.smithi053.stdout:4447 read 0x4b2a088 thru 0x4b2ff46 (0x5ebf bytes) 2024-09-13T05:02:56.920 INFO:teuthology.orchestra.run.smithi053.stdout:4448 read 0x3d21e36 thru 0x3d28330 (0x64fb bytes) 2024-09-13T05:02:56.922 INFO:teuthology.orchestra.run.smithi053.stdout:4449 trunc from 0xac1db55 to 0xce823a0 2024-09-13T05:02:56.931 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:02:56.931 INFO:teuthology.orchestra.run.smithi053.stdout:4450 write 0xacff55c thru 0xad05e8c (0x6931 bytes) 2024-09-13T05:02:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:4454 read 0xc3b2f41 thru 0xc3c0b94 (0xdc54 bytes) 2024-09-13T05:02:56.934 INFO:teuthology.orchestra.run.smithi053.stdout:4457 write 0x37f41a thru 0x38a13f (0xad26 bytes) 2024-09-13T05:02:56.959 INFO:teuthology.orchestra.run.smithi053.stdout:4460 trunc from 0xce823a0 to 0xcecdee2 2024-09-13T05:02:56.970 INFO:teuthology.orchestra.run.smithi053.stdout:4461 write 0xe70b938 thru 0xe7178e5 (0xbfae bytes) 2024-09-13T05:02:56.978 INFO:teuthology.orchestra.run.smithi053.stdout:4462 write 0x43f36ab thru 0x43f6f7e (0x38d4 bytes) 2024-09-13T05:02:57.013 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:56.975+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:57.016 INFO:teuthology.orchestra.run.smithi053.stdout:4463 punch from 0xb91e039 to 0xb92471a, (0x66e1 bytes) 2024-09-13T05:02:57.016 INFO:teuthology.orchestra.run.smithi053.stdout:4464 write 0x86b2f57 thru 0x86b99ea (0x6a94 bytes) 2024-09-13T05:02:57.022 INFO:teuthology.orchestra.run.smithi053.stdout:4468 write 0x1c0cafa thru 0x1c155c6 (0x8acd bytes) 2024-09-13T05:02:57.029 INFO:teuthology.orchestra.run.smithi053.stdout:4469 write 0x4d56547 thru 0x4d5d4b5 (0x6f6f bytes) 2024-09-13T05:02:57.034 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.044 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:02:57.045 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:02:57.045 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T05:02:57.045 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:02:57.045 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:02:57.046 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T05:02:57.046 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:57.047 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:57.047 INFO:teuthology.orchestra.run.smithi053.stdout:4472 trunc from 0xe7178e6 to 0x2c81729 2024-09-13T05:02:57.055 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:02:57.062 INFO:teuthology.orchestra.run.smithi053.stdout:4473 read 0x1af08a2 thru 0x1af26d9 (0x1e38 bytes) 2024-09-13T05:02:57.063 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:57.062+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:57.077 INFO:teuthology.orchestra.run.smithi053.stdout:4474 writesame 0xb40406b thru 0xb406598 data_size 0x1(0x252e bytes) 2024-09-13T05:02:57.077 INFO:teuthology.orchestra.run.smithi053.stdout:4475 punch from 0x6986beb to 0x698df03, (0x7318 bytes) 2024-09-13T05:02:57.077 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.078 INFO:teuthology.orchestra.run.smithi053.stdout:4477 trunc from 0xb406599 to 0xb6ebcf8 2024-09-13T05:02:57.092 INFO:teuthology.orchestra.run.smithi053.stdout:4478 trunc from 0xb6ebcf8 to 0x20370cc 2024-09-13T05:02:57.098 INFO:teuthology.orchestra.run.smithi053.stdout:4479 writesame 0xa5be1d thru 0xa5ff9d data_size 0x1(0x4181 bytes) 2024-09-13T05:02:57.117 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.117 INFO:teuthology.orchestra.run.smithi053.stdout:4481 read 0x24d0e2 thru 0x254aeb (0x7a0a bytes) 2024-09-13T05:02:57.126 INFO:teuthology.orchestra.run.smithi053.stdout:4482 writesame 0x43ab06c thru 0x43ba1d5 data_size 0xd(0xf16a bytes) 2024-09-13T05:02:57.126 INFO:teuthology.orchestra.run.smithi053.stdout:4484 punch from 0x3400416 to 0x340c9b2, (0xc59c bytes) 2024-09-13T05:02:57.126 INFO:teuthology.orchestra.run.smithi053.stdout:4486 read 0x437d53 thru 0x43de4f (0x60fd bytes) 2024-09-13T05:02:57.127 INFO:teuthology.orchestra.run.smithi053.stdout:4487 punch from 0x39ef86f to 0x39fdf50, (0xe6e1 bytes) 2024-09-13T05:02:57.127 INFO:teuthology.orchestra.run.smithi053.stdout:4488 read 0x4257cb1 thru 0x4265bdc (0xdf2c bytes) 2024-09-13T05:02:57.136 INFO:teuthology.orchestra.run.smithi053.stdout:4490 trunc from 0x43ba1d6 to 0xace502e 2024-09-13T05:02:57.140 INFO:teuthology.orchestra.run.smithi053.stdout:4491 writesame 0x6af2a92 thru 0x6af5018 data_size 0x1(0x2587 bytes) 2024-09-13T05:02:57.146 INFO:teuthology.orchestra.run.smithi053.stdout:4492 punch from 0x61ca760 to 0x61ce7db, (0x407b bytes) 2024-09-13T05:02:57.146 INFO:teuthology.orchestra.run.smithi053.stdout:4493 writesame 0x7c1ab1d thru 0x7c1c10c data_size 0x36(0x15f0 bytes) 2024-09-13T05:02:57.159 INFO:teuthology.orchestra.run.smithi053.stdout:4494 write 0xe379fb9 thru 0xe387d86 (0xddce bytes) 2024-09-13T05:02:57.164 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:57.163+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:57.168 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.169 INFO:teuthology.orchestra.run.smithi053.stdout:4496 write 0x38f48c2 thru 0x38fc389 (0x7ac8 bytes) 2024-09-13T05:02:57.170 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:57.170 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T05:02:57.171 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:57.175 INFO:teuthology.orchestra.run.smithi053.stdout:4497 write 0xc6b86ee thru 0xc6ba3b0 (0x1cc3 bytes) 2024-09-13T05:02:57.179 INFO:teuthology.orchestra.run.smithi053.stdout:4498 punch from 0x83bf1da to 0x83c0b1a, (0x1940 bytes) 2024-09-13T05:02:57.179 INFO:teuthology.orchestra.run.smithi053.stdout:4499 writesame 0xde83e51 thru 0xde91851 data_size 0xd(0xda01 bytes) 2024-09-13T05:02:57.189 INFO:teuthology.orchestra.run.smithi053.stdout:4500 writesame 0x388820 thru 0x38bfdc data_size 0x1(0x37bd bytes) 2024-09-13T05:02:57.194 INFO:teuthology.orchestra.run.smithi053.stdout:4501 read 0x182bc10 thru 0x183860b (0xc9fc bytes) 2024-09-13T05:02:57.195 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.195 INFO:teuthology.orchestra.run.smithi053.stdout:4505 write 0xbe26bc1 thru 0xbe2da7d (0x6ebd bytes) 2024-09-13T05:02:57.201 INFO:teuthology.orchestra.run.smithi053.stdout:4506 write 0x22c379e thru 0x22d11e3 (0xda46 bytes) 2024-09-13T05:02:57.211 INFO:teuthology.orchestra.run.smithi053.stdout:4507 punch from 0x5fd6a93 to 0x5fe616f, (0xf6dc bytes) 2024-09-13T05:02:57.211 INFO:teuthology.orchestra.run.smithi053.stdout:4508 writesame 0xad30954 thru 0xad3bb93 data_size 0x7c(0xb240 bytes) 2024-09-13T05:02:57.218 INFO:teuthology.orchestra.run.smithi053.stdout:4511 writesame 0x6236197 thru 0x6237167 data_size 0x1(0xfd1 bytes) 2024-09-13T05:02:57.220 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:57.220 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:57.222 INFO:teuthology.orchestra.run.smithi053.stdout:4512 write 0xb0555f thru 0xb10e32 (0xb8d4 bytes) 2024-09-13T05:02:57.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:02:57.244 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.244 INFO:teuthology.orchestra.run.smithi053.stdout:4515 writesame 0xddce1f thru 0xde23f9 data_size 0x1(0x55db bytes) 2024-09-13T05:02:57.253 INFO:teuthology.orchestra.run.smithi053.stdout:4516 read 0x6c1ff78 thru 0x6c25781 (0x580a bytes) 2024-09-13T05:02:57.254 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.254 INFO:teuthology.orchestra.run.smithi053.stdout:4518 write 0x10c1226 thru 0x10c7c79 (0x6a54 bytes) 2024-09-13T05:02:57.259 INFO:teuthology.orchestra.run.smithi053.stdout:4519 write 0x99b0910 thru 0x99b2596 (0x1c87 bytes) 2024-09-13T05:02:57.269 INFO:teuthology.orchestra.run.smithi053.stdout:4520 trunc from 0xe387d87 to 0x8b3b525 2024-09-13T05:02:57.273 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:57.272+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:57.283 INFO:teuthology.orchestra.run.smithi053.stdout:4521 writesame 0x3fa0355 thru 0x3fa0c6d data_size 0x1(0x919 bytes) 2024-09-13T05:02:57.287 INFO:teuthology.orchestra.run.smithi053.stdout:4524 write 0x481ff7a thru 0x4823da5 (0x3e2c bytes) 2024-09-13T05:02:57.291 INFO:teuthology.orchestra.run.smithi053.stdout:4525 read 0x4d190c0 thru 0x4d2549a (0xc3db bytes) 2024-09-13T05:02:57.294 INFO:teuthology.orchestra.run.smithi053.stdout:4526 trunc from 0x8b3b525 to 0x65db43b 2024-09-13T05:02:57.303 INFO:teuthology.orchestra.run.smithi053.stdout:4527 write 0x3a39546 thru 0x3a49213 (0xfcce bytes) 2024-09-13T05:02:57.310 INFO:teuthology.orchestra.run.smithi053.stdout:4529 trunc from 0x65db43b to 0x67f8259 2024-09-13T05:02:57.315 INFO:teuthology.orchestra.run.smithi053.stdout:4530 punch from 0x1cbab97 to 0x1cc1221, (0x668a bytes) 2024-09-13T05:02:57.315 INFO:teuthology.orchestra.run.smithi053.stdout:4532 read 0xc4a78a thru 0xc4d8a2 (0x3119 bytes) 2024-09-13T05:02:57.315 INFO:teuthology.orchestra.run.smithi053.stdout:4535 punch from 0x4c281e5 to 0x4c29086, (0xea1 bytes) 2024-09-13T05:02:57.330 INFO:teuthology.orchestra.run.smithi053.stdout:4536 writesame 0xe86d1df thru 0xe879007 data_size 0x1b(0xbe29 bytes) 2024-09-13T05:02:57.336 INFO:teuthology.orchestra.run.smithi053.stdout:4538 writesame 0xed43bcb thru 0xed46584 data_size 0x7(0x29ba bytes) 2024-09-13T05:02:57.336 INFO:teuthology.orchestra.run.smithi053.stdout:4539 write 0xb82eafb thru 0xb8394e2 (0xa9e8 bytes) 2024-09-13T05:02:57.343 INFO:teuthology.orchestra.run.smithi053.stdout:4540 read 0x3934a35 thru 0x39374c6 (0x2a92 bytes) 2024-09-13T05:02:57.345 INFO:teuthology.orchestra.run.smithi053.stdout:4541 trunc from 0xed46585 to 0xd6da9d0 2024-09-13T05:02:57.509 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:57.509 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:57.509 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:02:57.510 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:57.507+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:57.512 INFO:teuthology.orchestra.run.smithi053.stdout:4542 writesame 0xb3e3361 thru 0xb3f15ff data_size 0x29(0xe29f bytes) 2024-09-13T05:02:57.512 INFO:teuthology.orchestra.run.smithi053.stdout:4543 read 0xa4fc66a thru 0xa5086d9 (0xc070 bytes) 2024-09-13T05:02:57.512 INFO:teuthology.orchestra.run.smithi053.stdout:4544 write 0x58ca276 thru 0x58cf763 (0x54ee bytes) 2024-09-13T05:02:57.512 INFO:teuthology.orchestra.run.smithi053.stdout:4545 write 0xae63f2e thru 0xae6f161 (0xb234 bytes) 2024-09-13T05:02:57.512 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4549 read 0xc989a83 thru 0xc98fae0 (0x605e bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4550 trunc from 0xd6da9d0 to 0x765df0c 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4551 trunc from 0x765df0c to 0xd506ccf 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4553 read 0x67069b0 thru 0x670f77c (0x8dcd bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4554 punch from 0x1a6e8c8 to 0x1a7d286, (0xe9be bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4555 read 0x70dee60 thru 0x70e08f4 (0x1a95 bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4556 trunc from 0xd506ccf to 0x91f5aef 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4557 read 0x77d4231 thru 0x77d9dd9 (0x5ba9 bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4558 write 0x9ad14ec thru 0x9ad35e7 (0x20fc bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4559 trunc from 0x9ad35e8 to 0xb77788d 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4560 writesame 0x7bcd491 thru 0x7bd1736 data_size 0x13(0x42a6 bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4561 read 0x871f53c thru 0x8723539 (0x3ffe bytes) 2024-09-13T05:02:57.513 INFO:teuthology.orchestra.run.smithi053.stdout:4563 read 0x3a7b76b thru 0x3a881e6 (0xca7c bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4566 read 0x374d46b thru 0x37597c6 (0xc35c bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4567 punch from 0x6e5972c to 0x6e69071, (0xf945 bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4571 read 0x9baef4e thru 0x9bbb0c1 (0xc174 bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4572 read 0x6a7b0d2 thru 0x6a8068e (0x55bd bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4574 punch from 0x6a799fd to 0x6a7f1e6, (0x57e9 bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4576 read 0x183ce2 thru 0x18c63f (0x895e bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4577 read 0x3908738 thru 0x39111f2 (0x8abb bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:4578 punch from 0x48ce686 to 0x48cf21a, (0xb94 bytes) 2024-09-13T05:02:57.514 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4581 read 0x6e988fc thru 0x6e9bcea (0x33ef bytes) 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4582 writesame 0xf7d454 thru 0xf8b3ff data_size 0x14(0xdfac bytes) 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4584 write 0xa963f24 thru 0xa96e7ca (0xa8a7 bytes) 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4586 read 0x9527c2 thru 0x95f208 (0xca47 bytes) 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4587 write 0x7f8ba0e thru 0x7f8dd41 (0x2334 bytes) 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4589 trunc from 0xb77788d to 0x1b7f353 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4592 read 0x169aee3 thru 0x169e4ba (0x35d8 bytes) 2024-09-13T05:02:57.515 INFO:teuthology.orchestra.run.smithi053.stdout:4594 write 0x8c42410 thru 0x8c44f41 (0x2b32 bytes) 2024-09-13T05:02:57.516 INFO:teuthology.orchestra.run.smithi053.stdout:4595 write 0xc6df44d thru 0xc6eee7b (0xfa2f bytes) 2024-09-13T05:02:57.526 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.526 INFO:teuthology.orchestra.run.smithi053.stdout:4598 punch from 0x11751ef to 0x117e564, (0x9375 bytes) 2024-09-13T05:02:57.526 INFO:teuthology.orchestra.run.smithi053.stdout:4599 read 0x4ebd9f thru 0x4f7500 (0xb762 bytes) 2024-09-13T05:02:57.533 INFO:teuthology.orchestra.run.smithi053.stdout:4601 writesame 0xd1a98b4 thru 0xd1b212f data_size 0xa(0x887c bytes) 2024-09-13T05:02:57.540 INFO:teuthology.orchestra.run.smithi053.stdout:4602 write 0xea22fdd thru 0xea2a8c8 (0x78ec bytes) 2024-09-13T05:02:57.547 INFO:teuthology.orchestra.run.smithi053.stdout:4603 read 0x827ef6d thru 0x8286980 (0x7a14 bytes) 2024-09-13T05:02:57.547 INFO:teuthology.orchestra.run.smithi053.stdout:4604 read 0x1add4c6 thru 0x1ae2d30 (0x586b bytes) 2024-09-13T05:02:57.547 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.547 INFO:teuthology.orchestra.run.smithi053.stdout:4606 read 0xa64106d thru 0xa650cb6 (0xfc4a bytes) 2024-09-13T05:02:57.547 INFO:teuthology.orchestra.run.smithi053.stdout:4607 write 0xe58aab2 thru 0xe597c61 (0xd1b0 bytes) 2024-09-13T05:02:57.555 INFO:teuthology.orchestra.run.smithi053.stdout:4608 read 0xddc3a05 thru 0xddcd430 (0x9a2c bytes) 2024-09-13T05:02:57.556 INFO:teuthology.orchestra.run.smithi053.stdout:4609 write 0x5ab5e25 thru 0x5ab72d4 (0x14b0 bytes) 2024-09-13T05:02:57.559 INFO:teuthology.orchestra.run.smithi053.stdout:4610 write 0x653811 thru 0x65acf4 (0x74e4 bytes) 2024-09-13T05:02:57.583 INFO:teuthology.orchestra.run.smithi053.stdout:4611 read 0x4b9af0b thru 0x4ba5b39 (0xac2f bytes) 2024-09-13T05:02:57.583 INFO:teuthology.orchestra.run.smithi053.stdout:4612 write 0xcab3e5 thru 0xcaeef1 (0x3b0d bytes) 2024-09-13T05:02:57.589 INFO:teuthology.orchestra.run.smithi053.stdout:4613 write 0xcf6860d thru 0xcf76e4b (0xe83f bytes) 2024-09-13T05:02:57.599 INFO:teuthology.orchestra.run.smithi053.stdout:4614 write 0x19184c thru 0x19c0df (0xa894 bytes) 2024-09-13T05:02:57.608 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:57.608+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:57.627 INFO:teuthology.orchestra.run.smithi053.stdout:4615 write 0x934b3ee thru 0x934fbdd (0x47f0 bytes) 2024-09-13T05:02:57.632 INFO:teuthology.orchestra.run.smithi053.stdout:4616 read 0x364ff2b thru 0x365bec2 (0xbf98 bytes) 2024-09-13T05:02:57.632 INFO:teuthology.orchestra.run.smithi053.stdout:4617 write 0xa08a5fe thru 0xa08dbbe (0x35c1 bytes) 2024-09-13T05:02:57.636 INFO:teuthology.orchestra.run.smithi053.stdout:4618 punch from 0xca432e4 to 0xca49c62, (0x697e bytes) 2024-09-13T05:02:57.636 INFO:teuthology.orchestra.run.smithi053.stdout:4619 punch from 0x3e40a72 to 0x3e476e9, (0x6c77 bytes) 2024-09-13T05:02:57.636 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:57.636 INFO:teuthology.orchestra.run.smithi053.stdout:4622 write 0x59ae616 thru 0x59b6f3d (0x8928 bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4625 read 0xd4ed086 thru 0xd4f29d1 (0x594c bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4626 read 0x787dd02 thru 0x7880542 (0x2841 bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4628 read 0x32e5360 thru 0x32f3404 (0xe0a5 bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4630 read 0x7deb318 thru 0x7dedd37 (0x2a20 bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4631 punch from 0x6b45766 to 0x6b51ea5, (0xc73f bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4633 punch from 0x737d82d to 0x7389e15, (0xc5e8 bytes) 2024-09-13T05:02:57.643 INFO:teuthology.orchestra.run.smithi053.stdout:4634 write 0x5056d6f thru 0x5063ff5 (0xd287 bytes) 2024-09-13T05:02:57.661 INFO:teuthology.orchestra.run.smithi053.stdout:4636 trunc from 0xea2a8c9 to 0x5efcae5 2024-09-13T05:02:57.673 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:57.673 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:57.680 INFO:teuthology.orchestra.run.smithi053.stdout:4638 writesame 0x52d00ba thru 0x52de5f5 data_size 0x1(0xe53c bytes) 2024-09-13T05:02:57.683 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:02:57.695 INFO:teuthology.orchestra.run.smithi053.stdout:4639 clone 30 order 18 su 32768 sc 13 2024-09-13T05:02:57.709 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:57.709+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:57.810 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:57.809+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:57.827 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:57.827 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:57.835 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:02:57.911 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:57.910+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:57.940 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:57.941 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:57.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:02:57.989 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone29 intact 2024-09-13T05:02:58.012 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:02:58.012+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:02:58.057 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:58.057 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:58.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:02:58.113 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:02:58.113+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:02:58.178 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:58.178 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:58.178 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-13T05:02:58.190 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:02:58.233 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:58.233+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:58.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:58.308 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:58.316 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:58.315+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:58.317 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:02:58.395 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.405 INFO:teuthology.orchestra.run.smithi053.stdout:4641 trunc from 0x5efcae5 to 0xe988238 2024-09-13T05:02:58.414 INFO:teuthology.orchestra.run.smithi053.stdout:4642 write 0x5e2700b thru 0x5e28bb1 (0x1ba7 bytes) 2024-09-13T05:02:58.416 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:58.416+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:58.419 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.419 INFO:teuthology.orchestra.run.smithi053.stdout:4645 write 0xf3a634 thru 0xf4437e (0x9d4b bytes) 2024-09-13T05:02:58.425 INFO:teuthology.orchestra.run.smithi053.stdout:4646 writesame 0x25c4b7b thru 0x25cc599 data_size 0x11(0x7a1f bytes) 2024-09-13T05:02:58.429 INFO:teuthology.orchestra.run.smithi053.stdout:4647 write 0xb71e85 thru 0xb7dfbe (0xc13a bytes) 2024-09-13T05:02:58.452 INFO:teuthology.orchestra.run.smithi053.stdout:4648 trunc from 0xe988238 to 0x23999b2 2024-09-13T05:02:58.478 INFO:teuthology.orchestra.run.smithi053.stdout:4650 trunc from 0x23999b2 to 0xbc8b9cd 2024-09-13T05:02:58.482 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:58.482 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:58.483 INFO:teuthology.orchestra.run.smithi053.stdout:4657 read 0x4f28a00 thru 0x4f328e5 (0x9ee6 bytes) 2024-09-13T05:02:58.486 INFO:teuthology.orchestra.run.smithi053.stdout:4658 trunc from 0xbc8b9cd to 0x8c24a0a 2024-09-13T05:02:58.492 INFO:teuthology.orchestra.run.smithi053.stdout:4659 punch from 0x1fc9c4 to 0x207e05, (0xb441 bytes) 2024-09-13T05:02:58.492 INFO:teuthology.orchestra.run.smithi053.stdout:4660 writesame 0x7ac4564 thru 0x7ace454 data_size 0x63(0x9ef1 bytes) 2024-09-13T05:02:58.495 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:02:58.500 INFO:teuthology.orchestra.run.smithi053.stdout:4662 write 0x8fb4490 thru 0x8fb8c6f (0x47e0 bytes) 2024-09-13T05:02:58.511 INFO:teuthology.orchestra.run.smithi053.stdout:4664 write 0xae7b815 thru 0xae89cbb (0xe4a7 bytes) 2024-09-13T05:02:58.517 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:58.517+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:58.526 INFO:teuthology.orchestra.run.smithi053.stdout:4665 write 0xe9d8de8 thru 0xe9e37eb (0xaa04 bytes) 2024-09-13T05:02:58.530 INFO:teuthology.orchestra.run.smithi053.stdout:4666 read 0xbf2f014 thru 0xbf3a590 (0xb57d bytes) 2024-09-13T05:02:58.530 INFO:teuthology.orchestra.run.smithi053.stdout:4667 read 0x3a6aaf4 thru 0x3a7722d (0xc73a bytes) 2024-09-13T05:02:58.530 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.530 INFO:teuthology.orchestra.run.smithi053.stdout:4669 read 0x780757c thru 0x78168d3 (0xf358 bytes) 2024-09-13T05:02:58.531 INFO:teuthology.orchestra.run.smithi053.stdout:4670 write 0x89fcbfd thru 0x8a0b26e (0xe672 bytes) 2024-09-13T05:02:58.536 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.547 INFO:teuthology.orchestra.run.smithi053.stdout:4673 trunc from 0xe9e37ec to 0x4c1f108 2024-09-13T05:02:58.569 INFO:teuthology.orchestra.run.smithi053.stdout:4674 writesame 0x937e5a6 thru 0x937fc9e data_size 0x1(0x16f9 bytes) 2024-09-13T05:02:58.569 INFO:teuthology.orchestra.run.smithi053.stdout:4675 write 0x44b7240 thru 0x44bd4db (0x629c bytes) 2024-09-13T05:02:58.573 INFO:teuthology.orchestra.run.smithi053.stdout:4676 read 0x701c9a5 thru 0x702a680 (0xdcdc bytes) 2024-09-13T05:02:58.573 INFO:teuthology.orchestra.run.smithi053.stdout:4677 read 0xd3ab51 thru 0xd496d2 (0xeb82 bytes) 2024-09-13T05:02:58.574 INFO:teuthology.orchestra.run.smithi053.stdout:4679 punch from 0x5c528df to 0x5c54130, (0x1851 bytes) 2024-09-13T05:02:58.574 INFO:teuthology.orchestra.run.smithi053.stdout:4680 writesame 0x6a6252c thru 0x6a6bad7 data_size 0x7c(0x95ac bytes) 2024-09-13T05:02:58.577 INFO:teuthology.orchestra.run.smithi053.stdout:4681 read 0xdb7aa0 thru 0xdbb238 (0x3799 bytes) 2024-09-13T05:02:58.578 INFO:teuthology.orchestra.run.smithi053.stdout:4682 write 0x428f35 thru 0x433a53 (0xab1f bytes) 2024-09-13T05:02:58.588 INFO:teuthology.orchestra.run.smithi053.stdout:4684 read 0x61efe9e thru 0x61f8488 (0x85eb bytes) 2024-09-13T05:02:58.588 INFO:teuthology.orchestra.run.smithi053.stdout:4685 write 0x210b97c thru 0x210d088 (0x170d bytes) 2024-09-13T05:02:58.591 INFO:teuthology.orchestra.run.smithi053.stdout:4686 punch from 0x66c3a89 to 0x66cf61b, (0xbb92 bytes) 2024-09-13T05:02:58.591 INFO:teuthology.orchestra.run.smithi053.stdout:4687 read 0x69db thru 0xca12 (0x6038 bytes) 2024-09-13T05:02:58.591 INFO:teuthology.orchestra.run.smithi053.stdout:4688 write 0x2744f45 thru 0x275117d (0xc239 bytes) 2024-09-13T05:02:58.596 INFO:teuthology.orchestra.run.smithi053.stdout:4689 punch from 0x579dbbe to 0x57a975e, (0xbba0 bytes) 2024-09-13T05:02:58.596 INFO:teuthology.orchestra.run.smithi053.stdout:4690 write 0x1180231 thru 0x118f74d (0xf51d bytes) 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:4693 punch from 0x38016c1 to 0x380f65f, (0xdf9e bytes) 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:4694 read 0x26cb6ba thru 0x26cf932 (0x4279 bytes) 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:4695 punch from 0x14db7cd to 0x14ea3a2, (0xebd5 bytes) 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:4697 read 0x62ca9d5 thru 0x62ccc0b (0x2237 bytes) 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:4698 read 0x7e69ed1 thru 0x7e75577 (0xb6a7 bytes) 2024-09-13T05:02:58.603 INFO:teuthology.orchestra.run.smithi053.stdout:4699 read 0x184ba13 thru 0x184df7d (0x256b bytes) 2024-09-13T05:02:58.604 INFO:teuthology.orchestra.run.smithi053.stdout:4700 flatten 2024-09-13T05:02:58.619 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:58.619+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:58.648 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:58.648 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:58.660 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:02:58.744 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:58.744+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:58.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:02:58.814 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:02:58.814 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:02:58.821 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:58.820+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:58.823 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:02:58.871 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:02:58.871 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:02:58.889 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:02:58.890 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:02:58.899 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:02:58.906 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:02:58.906 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:02:58.922 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:58.921+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:58.930 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:02:58.930 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:02:58.941 INFO:teuthology.orchestra.run.smithi053.stdout:4701 read 0x752b1f9 thru 0x75345e7 (0x93ef bytes) 2024-09-13T05:02:58.941 INFO:teuthology.orchestra.run.smithi053.stdout:4703 read 0x5f23 thru 0xda70 (0x7b4e bytes) 2024-09-13T05:02:58.941 INFO:teuthology.orchestra.run.smithi053.stdout:4704 read 0x844f547 thru 0x8459663 (0xa11d bytes) 2024-09-13T05:02:58.941 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.949 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:02:58.949 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:02:58.949 INFO:teuthology.orchestra.run.smithi053.stdout:4706 write 0xa83b2bf thru 0xa840f2c (0x5c6e bytes) 2024-09-13T05:02:58.954 INFO:teuthology.orchestra.run.smithi053.stdout:4709 write 0x76da330 thru 0x76e2725 (0x83f6 bytes) 2024-09-13T05:02:58.959 INFO:teuthology.orchestra.run.smithi053.stdout:4710 write 0x6a8ce52 thru 0x6a8fd0c (0x2ebb bytes) 2024-09-13T05:02:58.962 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:58.972 INFO:teuthology.orchestra.run.smithi053.stdout:4712 trunc from 0xa840f2d to 0x293e037 2024-09-13T05:02:58.982 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:02:58.982 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:02:58.985 INFO:teuthology.orchestra.run.smithi053.stdout:4714 punch from 0x2570bc0 to 0x2572973, (0x1db3 bytes) 2024-09-13T05:02:58.985 INFO:teuthology.orchestra.run.smithi053.stdout:4716 read 0x51a355 thru 0x51bb87 (0x1833 bytes) 2024-09-13T05:02:59.000 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:59.000 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.63675142, 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:02:59.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.006 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636728377000001, 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.007 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:02:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:02:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:02:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:02:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.012 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636711044999998, 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.013 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:59.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:02:59.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:02:59.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636694501000001, 2024-09-13T05:02:59.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:02:59.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.023 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636677376000002, 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.024 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:02:59.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:59.030 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636661148000002, 2024-09-13T05:02:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:02:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.031 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:02:59.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:02:59.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:02:59.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636644337, 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.037 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:02:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:02:59.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:02:59.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:02:59.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.042 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636627719, 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.043 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:02:59.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:02:59.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636611140999999, 2024-09-13T05:02:59.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636574787000001, 2024-09-13T05:02:59.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.053 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:02:59.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636555843, 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.057 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:02:59.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.061 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636539208999999, 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:59.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.064 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.065 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636513208, 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.066 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:02:59.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:59.070 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636481269000001, 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:02:59.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636462481999999, 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.075 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:02:59.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:02:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:02:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636446362000001, 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.084 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636424301000002, 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.085 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:02:59.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.636407336000001, 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:02:59.089 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:02:59.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.324372937, 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:02:59.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.324353829, 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:02:59.098 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:02:59.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.102 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:59.103 INFO:teuthology.orchestra.run.smithi053.stdout:4717 writesame 0xbc7e293 thru 0xbc88ab5 data_size 0x5b(0xa823 bytes) 2024-09-13T05:02:59.103 INFO:teuthology.orchestra.run.smithi053.stdout:4720 trunc from 0xbc88ab6 to 0x8c0baed 2024-09-13T05:02:59.104 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:02:59.104 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:02:59.105 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:02:59.106 INFO:teuthology.orchestra.run.smithi053.stdout:4722 writesame 0x1b50fb3 thru 0x1b5c9f7 data_size 0x55(0xba45 bytes) 2024-09-13T05:02:59.106 INFO:teuthology.orchestra.run.smithi053.stdout:4723 read 0x629fd3e thru 0x62a3457 (0x371a bytes) 2024-09-13T05:02:59.106 INFO:teuthology.orchestra.run.smithi053.stdout:4724 write 0x8c20de6 thru 0x8c2dc45 (0xce60 bytes) 2024-09-13T05:02:59.106 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:02:59.106 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:02:59.107 INFO:teuthology.orchestra.run.smithi053.stdout:4725 write 0x974d9c2 thru 0x97540da (0x6719 bytes) 2024-09-13T05:02:59.107 INFO:teuthology.orchestra.run.smithi053.stdout:4726 trunc from 0x97540db to 0xe5d017d 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4727 read 0x9227330 thru 0x922c55c (0x522d bytes) 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4728 trunc from 0xe5d017d to 0x2e68918 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4730 writesame 0x5f8584d thru 0x5f908d1 data_size 0x3(0xb085 bytes) 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4731 trunc from 0x5f908d2 to 0x5ff7e06 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4732 write 0x1e6a74b thru 0x1e6e51c (0x3dd2 bytes) 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4734 read 0x2cfe6bd thru 0x2cfedee (0x732 bytes) 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4735 read 0xd90ec8 thru 0xd9ef78 (0xe0b1 bytes) 2024-09-13T05:02:59.109 INFO:teuthology.orchestra.run.smithi053.stdout:4736 write 0xd08ddc2 thru 0xd094755 (0x6994 bytes) 2024-09-13T05:02:59.110 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:02:59.103+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:02:59.110 INFO:teuthology.orchestra.run.smithi053.stdout:4738 write 0xa5bb3b8 thru 0xa5c06bd (0x5306 bytes) 2024-09-13T05:02:59.118 INFO:teuthology.orchestra.run.smithi053.stdout:4739 trunc from 0xd094756 to 0x9f1e3f3 2024-09-13T05:02:59.129 INFO:teuthology.orchestra.run.smithi053.stdout:4740 punch from 0x7eb5749 to 0x7eb82a6, (0x2b5d bytes) 2024-09-13T05:02:59.129 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.129 INFO:teuthology.orchestra.run.smithi053.stdout:4743 punch from 0x2ae804f to 0x2af1e2b, (0x9ddc bytes) 2024-09-13T05:02:59.129 INFO:teuthology.orchestra.run.smithi053.stdout:4744 punch from 0x7718249 to 0x771fa1a, (0x77d1 bytes) 2024-09-13T05:02:59.129 INFO:teuthology.orchestra.run.smithi053.stdout:4745 writesame 0xad7415 thru 0xae7242 data_size 0x5a(0xfe2e bytes) 2024-09-13T05:02:59.143 INFO:teuthology.orchestra.run.smithi053.stdout:4746 trunc from 0x9f1e3f3 to 0x21dde2b 2024-09-13T05:02:59.158 INFO:teuthology.orchestra.run.smithi053.stdout:4747 trunc from 0x21dde2b to 0xbb82ae 2024-09-13T05:02:59.180 INFO:teuthology.orchestra.run.smithi053.stdout:4749 punch from 0x1ec1eb to 0x1f923e, (0xd053 bytes) 2024-09-13T05:02:59.180 INFO:teuthology.orchestra.run.smithi053.stdout:4750 read 0x77c796 thru 0x788b35 (0xc3a0 bytes) 2024-09-13T05:02:59.184 INFO:teuthology.orchestra.run.smithi053.stdout:4751 write 0x1c2a806 thru 0x1c2f047 (0x4842 bytes) 2024-09-13T05:02:59.204 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:02:59.203+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:02:59.209 INFO:teuthology.orchestra.run.smithi053.stdout:4752 write 0xe5a1aa3 thru 0xe5ad031 (0xb58f bytes) 2024-09-13T05:02:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:4754 punch from 0x777d2e3 to 0x7787eae, (0xabcb bytes) 2024-09-13T05:02:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:4755 punch from 0xc18a983 to 0xc19a0a0, (0xf71d bytes) 2024-09-13T05:02:59.214 INFO:teuthology.orchestra.run.smithi053.stdout:4756 write 0x2f638c2 thru 0x2f72b83 (0xf2c2 bytes) 2024-09-13T05:02:59.219 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.231 INFO:teuthology.orchestra.run.smithi053.stdout:4758 trunc from 0xe5ad032 to 0x4c1e890 2024-09-13T05:02:59.259 INFO:teuthology.orchestra.run.smithi053.stdout:4759 write 0xd562223 thru 0xd563b6f (0x194d bytes) 2024-09-13T05:02:59.262 INFO:teuthology.orchestra.run.smithi053.stdout:4761 punch from 0x794dc5b to 0x7956b8f, (0x8f34 bytes) 2024-09-13T05:02:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:4763 read 0xacbdc74 thru 0xacc8cd0 (0xb05d bytes) 2024-09-13T05:02:59.263 INFO:teuthology.orchestra.run.smithi053.stdout:4764 writesame 0xa3be8eb thru 0xa3c2775 data_size 0x9(0x3e8b bytes) 2024-09-13T05:02:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:59.266 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:59.266 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:59.266 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:4765 read 0x3402f73 thru 0x340329d (0x32b bytes) 2024-09-13T05:02:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:4766 read 0xa88541b thru 0xa8867dc (0x13c2 bytes) 2024-09-13T05:02:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:4767 read 0x589360b thru 0x589c3f3 (0x8de9 bytes) 2024-09-13T05:02:59.267 INFO:teuthology.orchestra.run.smithi053.stdout:4768 read 0xbf16b31 thru 0xbf17dde (0x12ae bytes) 2024-09-13T05:02:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:4769 read 0x57ba4c5 thru 0x57bc2c4 (0x1e00 bytes) 2024-09-13T05:02:59.268 INFO:teuthology.orchestra.run.smithi053.stdout:4771 write 0x99b47b8 thru 0x99c2bd4 (0xe41d bytes) 2024-09-13T05:02:59.272 INFO:teuthology.orchestra.run.smithi053.stdout:4772 read 0xd2f5e07 thru 0xd2f7544 (0x173e bytes) 2024-09-13T05:02:59.279 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:02:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:4773 writesame 0xedc5672 thru 0xedd48e6 data_size 0x1(0xf275 bytes) 2024-09-13T05:02:59.280 INFO:teuthology.orchestra.run.smithi053.stdout:4776 write 0x58e3c46 thru 0x58ee49b (0xa856 bytes) 2024-09-13T05:02:59.284 INFO:teuthology.orchestra.run.smithi053.stdout:4777 write 0x20135f thru 0x2047af (0x3451 bytes) 2024-09-13T05:02:59.286 INFO:teuthology.orchestra.run.smithi053.stdout:4778 write 0x26af7d4 thru 0x26bc8ac (0xd0d9 bytes) 2024-09-13T05:02:59.291 INFO:teuthology.orchestra.run.smithi053.stdout:4779 punch from 0xc8e47a9 to 0xc8eca44, (0x829b bytes) 2024-09-13T05:02:59.300 INFO:teuthology.orchestra.run.smithi053.stdout:4780 trunc from 0xedd48e7 to 0x6dc4491 2024-09-13T05:02:59.304 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:59.304+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:4781 read 0x35f3fbe thru 0x35f626a (0x22ad bytes) 2024-09-13T05:02:59.312 INFO:teuthology.orchestra.run.smithi053.stdout:4782 read 0x21ed001 thru 0x21fbfe6 (0xefe6 bytes) 2024-09-13T05:02:59.318 INFO:teuthology.orchestra.run.smithi053.stdout:4783 write 0x81b6573 thru 0x81c2ccc (0xc75a bytes) 2024-09-13T05:02:59.330 INFO:teuthology.orchestra.run.smithi053.stdout:4785 write 0x938d3d0 thru 0x93955fc (0x822d bytes) 2024-09-13T05:02:59.334 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.339 INFO:teuthology.orchestra.run.smithi053.stdout:4787 trunc from 0x93955fd to 0x49729c1 2024-09-13T05:02:59.362 INFO:teuthology.orchestra.run.smithi053.stdout:4790 write 0xa4388b7 thru 0xa445b0b (0xd255 bytes) 2024-09-13T05:02:59.366 INFO:teuthology.orchestra.run.smithi053.stdout:4792 punch from 0x96d3863 to 0x96dca8b, (0x9228 bytes) 2024-09-13T05:02:59.366 INFO:teuthology.orchestra.run.smithi053.stdout:4793 write 0x9755a48 thru 0x975dd3f (0x82f8 bytes) 2024-09-13T05:02:59.378 INFO:teuthology.orchestra.run.smithi053.stdout:4794 trunc from 0xa445b0c to 0x33ca052 2024-09-13T05:02:59.394 INFO:teuthology.orchestra.run.smithi053.stdout:4796 trunc from 0x33ca052 to 0x74dab4b 2024-09-13T05:02:59.399 INFO:teuthology.orchestra.run.smithi053.stdout:4798 read 0x514bd7c thru 0x514e248 (0x24cd bytes) 2024-09-13T05:02:59.399 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.405 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:59.405+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:59.407 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:59.407 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:59.408 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:59.408 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:59.408 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:59.412 INFO:teuthology.orchestra.run.smithi053.stdout:4800 write 0xe3e96a2 thru 0xe3f4f90 (0xb8ef bytes) 2024-09-13T05:02:59.416 INFO:teuthology.orchestra.run.smithi053.stdout:4801 punch from 0x562cdaa to 0x5635abe, (0x8d14 bytes) 2024-09-13T05:02:59.416 INFO:teuthology.orchestra.run.smithi053.stdout:4805 write 0xb827208 thru 0xb82bed7 (0x4cd0 bytes) 2024-09-13T05:02:59.418 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:02:59.420 INFO:teuthology.orchestra.run.smithi053.stdout:4806 read 0xa4865ce thru 0xa491477 (0xaeaa bytes) 2024-09-13T05:02:59.420 INFO:teuthology.orchestra.run.smithi053.stdout:4807 punch from 0xa195902 to 0xa1a10ac, (0xb7aa bytes) 2024-09-13T05:02:59.420 INFO:teuthology.orchestra.run.smithi053.stdout:4808 read 0x2d70706 thru 0x2d74a72 (0x436d bytes) 2024-09-13T05:02:59.423 INFO:teuthology.orchestra.run.smithi053.stdout:4809 trunc from 0xe3f4f91 to 0xb47ec05 2024-09-13T05:02:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:4810 read 0x2a8f0e2 thru 0x2a99478 (0xa397 bytes) 2024-09-13T05:02:59.434 INFO:teuthology.orchestra.run.smithi053.stdout:4811 punch from 0xb184963 to 0xb19278a, (0xde27 bytes) 2024-09-13T05:02:59.437 INFO:teuthology.orchestra.run.smithi053.stdout:4812 trunc from 0xb47ec05 to 0xdd0be30 2024-09-13T05:02:59.443 INFO:teuthology.orchestra.run.smithi053.stdout:4813 punch from 0x8dd56a0 to 0x8de4061, (0xe9c1 bytes) 2024-09-13T05:02:59.443 INFO:teuthology.orchestra.run.smithi053.stdout:4814 write 0x65a44b thru 0x662289 (0x7e3f bytes) 2024-09-13T05:02:59.445 INFO:teuthology.orchestra.run.smithi053.stdout:4815 read 0xbb571e8 thru 0xbb5c209 (0x5022 bytes) 2024-09-13T05:02:59.445 INFO:teuthology.orchestra.run.smithi053.stdout:4816 punch from 0x2617355 to 0x262655b, (0xf206 bytes) 2024-09-13T05:02:59.448 INFO:teuthology.orchestra.run.smithi053.stdout:4817 trunc from 0xdd0be30 to 0xac3a10a 2024-09-13T05:02:59.453 INFO:teuthology.orchestra.run.smithi053.stdout:4818 punch from 0x85f5a93 to 0x860111b, (0xb688 bytes) 2024-09-13T05:02:59.463 INFO:teuthology.orchestra.run.smithi053.stdout:4819 writesame 0xe95d6b3 thru 0xe965794 data_size 0x75(0x80e2 bytes) 2024-09-13T05:02:59.463 INFO:teuthology.orchestra.run.smithi053.stdout:4821 read 0x93352b8 thru 0x933adff (0x5b48 bytes) 2024-09-13T05:02:59.463 INFO:teuthology.orchestra.run.smithi053.stdout:4823 punch from 0xab1e279 to 0xab1fe1d, (0x1ba4 bytes) 2024-09-13T05:02:59.463 INFO:teuthology.orchestra.run.smithi053.stdout:4825 writesame 0x23c2c6b thru 0x23c4d01 data_size 0x1b(0x2097 bytes) 2024-09-13T05:02:59.465 INFO:teuthology.orchestra.run.smithi053.stdout:4829 writesame 0x96b7c8c thru 0x96c696d data_size 0x1b(0xece2 bytes) 2024-09-13T05:02:59.476 INFO:teuthology.orchestra.run.smithi053.stdout:4831 writesame 0xecb844a thru 0xecc54f5 data_size 0xa(0xd0ac bytes) 2024-09-13T05:02:59.476 INFO:teuthology.orchestra.run.smithi053.stdout:4832 writesame 0x183fd00 thru 0x184c85e data_size 0x1(0xcb5f bytes) 2024-09-13T05:02:59.496 INFO:teuthology.orchestra.run.smithi053.stdout:4833 write 0x4915162 thru 0x491e368 (0x9207 bytes) 2024-09-13T05:02:59.500 INFO:teuthology.orchestra.run.smithi053.stdout:4835 write 0xdf65586 thru 0xdf6a318 (0x4d93 bytes) 2024-09-13T05:02:59.504 INFO:teuthology.orchestra.run.smithi053.stdout:4836 punch from 0x3f89d76 to 0x3f8ee13, (0x509d bytes) 2024-09-13T05:02:59.504 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.506 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:02:59.506+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:02:59.518 INFO:teuthology.orchestra.run.smithi053.stdout:4838 trunc from 0xecc54f6 to 0x22f0fc5 2024-09-13T05:02:59.541 INFO:teuthology.orchestra.run.smithi053.stdout:4839 write 0x60c319c thru 0x60c520f (0x2074 bytes) 2024-09-13T05:02:59.543 INFO:teuthology.orchestra.run.smithi053.stdout:4840 read 0x19b5d42 thru 0x19bc6c5 (0x6984 bytes) 2024-09-13T05:02:59.544 INFO:teuthology.orchestra.run.smithi053.stdout:4841 read 0x4286bf8 thru 0x428e001 (0x740a bytes) 2024-09-13T05:02:59.544 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.544 INFO:teuthology.orchestra.run.smithi053.stdout:4844 read 0x3485972 thru 0x348a1a6 (0x4835 bytes) 2024-09-13T05:02:59.544 INFO:teuthology.orchestra.run.smithi053.stdout:4845 write 0x13e8db1 thru 0x13ea298 (0x14e8 bytes) 2024-09-13T05:02:59.546 INFO:teuthology.orchestra.run.smithi053.stdout:4847 punch from 0x3f847b1 to 0x3f887d8, (0x4027 bytes) 2024-09-13T05:02:59.546 INFO:teuthology.orchestra.run.smithi053.stdout:4848 read 0xcc0f98 thru 0xcc7735 (0x679e bytes) 2024-09-13T05:02:59.546 INFO:teuthology.orchestra.run.smithi053.stdout:4849 punch from 0x2b11508 to 0x2b16237, (0x4d2f bytes) 2024-09-13T05:02:59.551 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:59.551 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:02:59.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:02:59.551 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:02:59.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.571508428999998, 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:02:59.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:02:59.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:02:59.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.556 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10094 3.4 e29/19 3:24263961:::rbd_data.117c6c62fa22.000000000000000b:head v 29'95, mlcod=28'92)", 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.568437019999998, 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001767984, 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981135+0000", 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981152+0000", 2024-09-13T05:02:59.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6986999999999998e-05 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981166+0000", 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4331000000000001e-05 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981177+0000", 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0858e-05 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981233+0000", 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5813000000000002e-05 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.981253+0000", 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9707999999999998e-05 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982869+0000", 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001615932 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.982903+0000", 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4626999999999999e-05 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.561 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.746124374000001, 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:02:59.565 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.60505534, 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:02:59.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.37353469, 2024-09-13T05:02:59.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:02:59.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:02:59.574 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.372559017, 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:02:59.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:02:59.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.37223814, 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:02:59.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.583 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.370150183, 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.275841008, 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.589 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:02:59.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 21.970702546999998, 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.597 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:02:59.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:02:59.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:02:59.601 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52876 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'388, mlcod=44'385)", 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.211389144999998, 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922389, 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338182+0000", 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338190+0000", 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.345e-06 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338204+0000", 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4399999999999999e-05 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338212+0000", 2024-09-13T05:02:59.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3880000000000002e-06 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338273+0000", 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1030999999999999e-05 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338293+0000", 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472e-05 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340073+0000", 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017803719999999999 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340106+0000", 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2186e-05 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52877 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'389, mlcod=44'385)", 2024-09-13T05:02:59.606 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.210362831000001, 2024-09-13T05:02:59.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00170028, 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339210+0000", 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339209+0000", 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339215+0000", 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5540000000000003e-06 2024-09-13T05:02:59.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339226+0000", 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.112e-05 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339233+0000", 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7800000000000003e-06 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339568+0000", 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00033527999999999997 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.339596+0000", 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7784e-05 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340876+0000", 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012799409999999999 2024-09-13T05:02:59.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340910+0000", 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3596000000000002e-05 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52897 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'391, mlcod=44'388)", 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.205986813999999, 2024-09-13T05:02:59.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001898648, 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343585+0000", 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343592+0000", 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9110000000000004e-06 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343602+0000", 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:02:59.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343609+0000", 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9099999999999999e-06 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343673+0000", 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4048999999999996e-05 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343692+0000", 2024-09-13T05:02:59.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8946000000000001e-05 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345449+0000", 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017568970000000001 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345484+0000", 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5033999999999997e-05 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.616 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.191588472999999, 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:02:59.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:02:59.621 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.190301471000001, 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:02:59.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:02:59.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52997 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'399, mlcod=44'395)", 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.189105278, 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001838309, 2024-09-13T05:02:59.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360466+0000", 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360473+0000", 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6150000000000002e-06 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360484+0000", 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1549000000000001e-05 2024-09-13T05:02:59.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360495+0000", 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361021+0000", 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000526035 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361040+0000", 2024-09-13T05:02:59.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8482e-05 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362269+0000", 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012290960000000001 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362305+0000", 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6733000000000002e-05 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:53113 3.4 e44/19 3:2666468c:::rbd_data.117c12c41b07.0000000000000032:head v 44'404, mlcod=44'402)", 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 19.173273480999999, 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017669179999999999, 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376299+0000", 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376298+0000", 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376382+0000", 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3770999999999995e-05 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376397+0000", 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5688e-05 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376406+0000", 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3459999999999994e-06 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376476+0000", 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0240999999999994e-05 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.376502+0000", 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.5723000000000001e-05 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.378025+0000", 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001523169 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.378066+0000", 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1072999999999998e-05 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:81104 3.f 3:f399b833:::rbd_data.117c3247720a.000000000000025b:head [set-alloc-hint object_size 262144 write_size 262144,write 154050~9790] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:59.039685+0000", 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.50988722799999997, 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017099610000000001, 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 81104 2024-09-13T05:02:59.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039685+0000", 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039685+0000", 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039683+0000", 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039701+0000", 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8037e-05 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039722+0000", 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1325e-05 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039733+0000", 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0386e-05 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039806+0000", 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3468999999999997e-05 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.039954+0000", 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014767400000000001 2024-09-13T05:02:59.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.040094+0000", 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013988299999999999 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.040974+0000", 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00087987799999999995 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.041318+0000", 2024-09-13T05:02:59.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00034391899999999998 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.041357+0000", 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8989000000000002e-05 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.041395+0000", 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8557999999999999e-05 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:81162 3.f 3:f5174a4c:::rbd_data.117c3247720a.0000000000000344:head [set-alloc-hint object_size 262144 write_size 262144,write 56770~8766] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:59.108368+0000", 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.44120399100000002, 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017714950000000001, 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 81162 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108368+0000", 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108368+0000", 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108367+0000", 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:59.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108379+0000", 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2941e-05 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108397+0000", 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7441000000000001e-05 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108407+0000", 2024-09-13T05:02:59.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0297e-05 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108463+0000", 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6149000000000001e-05 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108624+0000", 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000160519 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:59.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.108782+0000", 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000158431 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.109631+0000", 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00084870599999999996 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.110051+0000", 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00042040399999999998 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.110104+0000", 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2679999999999997e-05 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.110140+0000", 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5746e-05 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:81537 3.f 3:f4075f82:::rbd_data.117c3247720a.0000000000000256:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:59.521224+0000", 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.028348273, 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017693679999999999, 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 81537 2024-09-13T05:02:59.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521224+0000", 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521224+0000", 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:02:59.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521222+0000", 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521235+0000", 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3087e-05 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521255+0000", 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0281000000000001e-05 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521266+0000", 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0382e-05 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521329+0000", 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3095000000000005e-05 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521430+0000", 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101018 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.521542+0000", 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000112429 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.522318+0000", 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00077582200000000001 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.522931+0000", 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00061340600000000004 2024-09-13T05:02:59.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.522961+0000", 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9957000000000001e-05 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:59.522993+0000", 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1980999999999998e-05 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:02:59.653 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:02:59.654 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:59.654 INFO:teuthology.orchestra.run.smithi053.stdout:4850 write 0x77931c3 thru 0x77a0ab7 (0xd8f5 bytes) 2024-09-13T05:02:59.655 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.656 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:02:59.657 INFO:teuthology.orchestra.run.smithi053.stdout:4852 trunc from 0x77a0ab8 to 0x98ab270 2024-09-13T05:02:59.657 INFO:teuthology.orchestra.run.smithi053.stdout:4853 punch from 0x13277a4 to 0x133156d, (0x9dc9 bytes) 2024-09-13T05:02:59.657 INFO:teuthology.orchestra.run.smithi053.stdout:4854 write 0x474b953 thru 0x4751930 (0x5fde bytes) 2024-09-13T05:02:59.657 INFO:teuthology.orchestra.run.smithi053.stdout:4855 write 0xd911796 thru 0xd91c7c5 (0xb030 bytes) 2024-09-13T05:02:59.657 INFO:teuthology.orchestra.run.smithi053.stdout:4856 write 0x3df96a4 thru 0x3e04e0c (0xb769 bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4860 trunc from 0xd91c7c6 to 0xc9b4fd 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4862 write 0xa7f71c7 thru 0xa7fdbde (0x6a18 bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4863 writesame 0x4d7f5fa thru 0x4d85fbf data_size 0x3(0x69c6 bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4866 writesame 0x21b5cc thru 0x21e095 data_size 0x2(0x2aca bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4871 read 0x40cb514 thru 0x40ce86a (0x3357 bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4872 write 0x63ba399 thru 0x63c17fc (0x7464 bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4875 read 0x3be2425 thru 0x3bed149 (0xad25 bytes) 2024-09-13T05:02:59.658 INFO:teuthology.orchestra.run.smithi053.stdout:4876 trunc from 0xa7fdbdf to 0xb9979b8 2024-09-13T05:02:59.658 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:59.654+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:02:59.660 INFO:teuthology.orchestra.run.smithi053.stdout:4877 trunc from 0xb9979b8 to 0xd038475 2024-09-13T05:02:59.665 INFO:teuthology.orchestra.run.smithi053.stdout:4878 write 0x7f1818d thru 0x7f27d2f (0xfba3 bytes) 2024-09-13T05:02:59.670 INFO:teuthology.orchestra.run.smithi053.stdout:4879 clone 31 order 22 su 32768 sc 9 2024-09-13T05:02:59.756 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:02:59.755+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:02:59.766 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:59.767 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:59.767 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:02:59.767 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:59.775 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:02:59.859 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:02:59.859+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:02:59.887 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:02:59.929 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:02:59.930 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:02:59.930 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:02:59.930 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:02:59.930 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:02:59.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:02:59.981 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:02:59.980+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:00.059 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:00.058+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:00.102 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.582789370999997, 2024-09-13T05:03:00.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:00.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:00.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:00.112 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:00.113 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.582618066000002, 2024-09-13T05:03:00.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:00.113 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.113 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:00.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.113 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.114 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:00.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.114 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:00.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:00.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:00.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:00.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:00.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:00.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:00.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:00.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:00.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:00.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:00.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:00.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:00.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:00.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:00.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:00.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:00.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:00.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:00.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:00.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:00.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:00.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.127 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.127 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.127 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:00.129 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.655677823, 2024-09-13T05:03:00.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:00.129 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.129 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.129 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:00.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:00.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.133 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.655668044, 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.134 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:00.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.138 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.6556540059999998, 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.139 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.6556278400000002, 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:00.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5507493879999998, 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:00.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5484953529999999, 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.153 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:00.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.548478979, 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:00.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.161 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.546648888, 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:00.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:00.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.546623764, 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:00.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:00.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.546613362, 2024-09-13T05:03:00.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:00.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:00.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5442949960000001, 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.544270413, 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:00.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:00.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:00.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:00.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:00.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:00.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:00.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.187 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:00.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5441874499999999, 2024-09-13T05:03:00.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:00.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.189 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.189 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:00.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:00.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:00.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:00.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:00.196 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5441668690000001, 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.197 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:00.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.203 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5441502950000001, 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:00.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:00.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5441332980000002, 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.212 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.542836613, 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:00.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 3.5428197290000001, 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:00.221 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:00.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.225 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:00.226 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:00.226 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:00.227 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:00.228 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-13T05:03:00.228 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:00.226+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:00.301 INFO:teuthology.orchestra.run.smithi053.stdout:4881 write 0xbc2c073 thru 0xbc35f2e (0x9ebc bytes) 2024-09-13T05:03:00.311 INFO:teuthology.orchestra.run.smithi053.stdout:4882 write 0xc1bbf8c thru 0xc1c800d (0xc082 bytes) 2024-09-13T05:03:00.316 INFO:teuthology.orchestra.run.smithi053.stdout:4883 writesame 0x42e1788 thru 0x42f0dd4 data_size 0x1(0xf64d bytes) 2024-09-13T05:03:00.328 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:00.327+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:00.334 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:00.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:00.335 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:00.335 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:00.338 INFO:teuthology.orchestra.run.smithi053.stdout:4884 write 0xcd16273 thru 0xcd21cce (0xba5c bytes) 2024-09-13T05:03:00.344 INFO:teuthology.orchestra.run.smithi053.stdout:4885 writesame 0x622a50c thru 0x622e6d0 data_size 0x1(0x41c5 bytes) 2024-09-13T05:03:00.345 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:00.353 INFO:teuthology.orchestra.run.smithi053.stdout:4886 read 0x78727c2 thru 0x787c390 (0x9bcf bytes) 2024-09-13T05:03:00.353 INFO:teuthology.orchestra.run.smithi053.stdout:4887 read 0x1c92ee1 thru 0x1c9be47 (0x8f67 bytes) 2024-09-13T05:03:00.353 INFO:teuthology.orchestra.run.smithi053.stdout:4888 read 0x5936b8f thru 0x593c185 (0x55f7 bytes) 2024-09-13T05:03:00.354 INFO:teuthology.orchestra.run.smithi053.stdout:4889 clone 32 order 20 su 524288 sc 3 2024-09-13T05:03:00.429 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:00.429+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:00.487 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:00.487 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:00.487 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:00.487 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:00.487 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:00.495 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:00.537 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:00.536+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:00.630 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:00.629+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:00.639 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:00.639 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:00.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395751021000002, 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.640 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:03:00.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.644 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395734765999997, 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:03:00.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:03:00.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395717677, 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395700810000001, 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.655 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:03:00.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:03:00.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395651512000001, 2024-09-13T05:03:00.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:03:00.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395628242999997, 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.665 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:03:00.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:03:00.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:03:00.670 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395612509000003, 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.671 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:03:00.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395589528999999, 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.676 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.681 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395540463000003, 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:03:00.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:03:00.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:03:00.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395419148000002, 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.687 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:00.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:03:00.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395363934000002, 2024-09-13T05:03:00.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:03:00.693 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.693 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.693 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.693 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:03:00.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395343644999997, 2024-09-13T05:03:00.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:03:00.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:03:00.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:03:00.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395326464, 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:03:00.703 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:03:00.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395238954, 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:03:00.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:00.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:03:00.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:03:00.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.714 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395219705999999, 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:03:00.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:00.719 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395158590999998, 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.720 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:03:00.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:03:00.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.725 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395141416000001, 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:00.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:03:00.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395125006000001, 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:03:00.730 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:03:00.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:03:00.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:03:00.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:03:00.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395104021999998, 2024-09-13T05:03:00.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:03:00.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:03:00.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:03:00.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 33.395086597999999, 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:00.742 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:00.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:00.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:03:00.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:03:00.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:00.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:00.748 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:00.749 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:00.750 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:00.748+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:00.850 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:00.849+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:00.851 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:00.852 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:00.852 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:00.852 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:00.860 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:00.951 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:00.950+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:00.958 INFO:teuthology.orchestra.run.smithi053.stdout:flattening image image_client.0-clone31 2024-09-13T05:03:00.982 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:00.982 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:00.982 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:00.982 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:00.983 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:00.994 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:01.052 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:01.052+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:01.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:01.134 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-13T05:03:01.140 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:01.140 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:01.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5222198010000003, 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:01.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.145 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5222012219999996, 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:01.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:01.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:01.150 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5221764279999999, 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.151 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:01.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5221570719999997, 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:01.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:01.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.522138386, 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:01.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5221133169999996, 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.164 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:01.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:01.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5220909420000002, 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:01.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5220701239999999, 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:01.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:01.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5220448199999996, 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:01.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5220260950000002, 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.189 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5220080339999997, 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:01.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.194 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5219758030000001, 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:01.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.198 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5219566430000002, 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.199 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:01.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5219395410000001, 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.203 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:01.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.207 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5219132440000003, 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.208 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:01.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:01.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5218873970000004, 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.212 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:01.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:01.216 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5218705449999996, 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:01.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5218541850000005, 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.221 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:01.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5218380610000004, 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:01.225 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.230 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 6.5218168309999998, 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:01.231 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:01.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:01.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.235 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:01.236 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:01.236 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:01.237 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:01.238 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:01.236+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:01.251 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:01.251 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:01.268 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:01.268 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:01.284 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:01.285 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:01.301 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:01.301 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:01.338 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:01.337+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:01.340 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:01.340 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:01.340 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:01.340 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:01.348 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:01.375 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:01.375 INFO:teuthology.orchestra.run.smithi053.stdout:4891 read 0x6b2b319 thru 0x6b356c0 (0xa3a8 bytes) 2024-09-13T05:03:01.381 INFO:teuthology.orchestra.run.smithi053.stdout:4893 trunc from 0xd038475 to 0x8a6e973 2024-09-13T05:03:01.393 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:01.393 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:01.393 INFO:teuthology.orchestra.run.smithi053.stdout:4899 write 0x57b57f0 thru 0x57be6ea (0x8efb bytes) 2024-09-13T05:03:01.397 INFO:teuthology.orchestra.run.smithi053.stdout:4900 punch from 0x830eaa6 to 0x831168f, (0x2be9 bytes) 2024-09-13T05:03:01.409 INFO:teuthology.orchestra.run.smithi053.stdout:4903 writesame 0xe72a611 thru 0xe733d54 data_size 0x2a(0x9744 bytes) 2024-09-13T05:03:01.410 INFO:teuthology.orchestra.run.smithi053.stdout:4904 write 0x2086281 thru 0x2095e37 (0xfbb7 bytes) 2024-09-13T05:03:01.414 INFO:teuthology.orchestra.run.smithi053.stdout:4905 write 0xe4bc57f thru 0xe4bfee5 (0x3967 bytes) 2024-09-13T05:03:01.418 INFO:teuthology.orchestra.run.smithi053.stdout:4907 read 0x2332b73 thru 0x233d7cd (0xac5b bytes) 2024-09-13T05:03:01.418 INFO:teuthology.orchestra.run.smithi053.stdout:4908 read 0x6dcd83 thru 0x6dd0e0 (0x35e bytes) 2024-09-13T05:03:01.418 INFO:teuthology.orchestra.run.smithi053.stdout:4909 read 0x4e10a8b thru 0x4e12789 (0x1cff bytes) 2024-09-13T05:03:01.419 INFO:teuthology.orchestra.run.smithi053.stdout:4910 writesame 0x31f3fd4 thru 0x31f9618 data_size 0x7(0x5645 bytes) 2024-09-13T05:03:01.423 INFO:teuthology.orchestra.run.smithi053.stdout:4911 clone 33 order 18 su 4096 sc 11 2024-09-13T05:03:01.438 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:01.438+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:01.492 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:01.493 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:01.493 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:01.501 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:01.519 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:01.519 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:01.536 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:01.536 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:01.540 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:01.539+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:01.552 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:01.552 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:01.568 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:01.569 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:01.641 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:01.640+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:01.642 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:01.642 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:01.651 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:01.741 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:01.741+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:01.795 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:01.795 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:01.804 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:01.843 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:01.842+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:01.904 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:03:01.948 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:01.948+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:01.952 INFO:teuthology.orchestra.run.smithi053.stdout:flattening image image_client.0-clone32 2024-09-13T05:03:01.956 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:01.956 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:01.970 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:02.045 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:02.044+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:02.091 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:02.091 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:02.100 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:02.145 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:02.145+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:02.182 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:02.182 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:02.182 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-13T05:03:02.182 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:03:02.182 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:03:02.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-13T05:03:02.188 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-13T05:03:02.218 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:02.218 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:02.231 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:02.273 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:02.273+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:02.304 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:02.304 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "" 2024-09-13T05:03:02.304 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:02.350 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:02.349+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:02.385 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:02.385 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:02.394 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:02.421 INFO:teuthology.orchestra.run.smithi053.stdout:4912 trunc from 0xe733d55 to 0xbb0ca5b 2024-09-13T05:03:02.441 INFO:teuthology.orchestra.run.smithi053.stdout:4913 writesame 0x64ee92d thru 0x64f7d33 data_size 0x41(0x9407 bytes) 2024-09-13T05:03:02.452 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:02.452+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:02.492 INFO:teuthology.orchestra.run.smithi053.stdout:4915 punch from 0x1e08edb to 0x1e092e1, (0x406 bytes) 2024-09-13T05:03:02.492 INFO:teuthology.orchestra.run.smithi053.stdout:4916 read 0x2ac6f25 thru 0x2ad3db6 (0xce92 bytes) 2024-09-13T05:03:02.499 INFO:teuthology.orchestra.run.smithi053.stdout:4917 writesame 0xccc1c0b thru 0xccc4154 data_size 0x6(0x254a bytes) 2024-09-13T05:03:02.499 INFO:teuthology.orchestra.run.smithi053.stdout:4918 read 0x4f0830 thru 0x4f805a (0x782b bytes) 2024-09-13T05:03:02.502 INFO:teuthology.orchestra.run.smithi053.stdout:4920 trunc from 0xccc4155 to 0xe86f827 2024-09-13T05:03:02.508 INFO:teuthology.orchestra.run.smithi053.stdout:4923 write 0x31edda6 thru 0x31fdac9 (0xfd24 bytes) 2024-09-13T05:03:02.522 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:02.522 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:02.530 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:02.530 INFO:teuthology.orchestra.run.smithi053.stdout:4928 punch from 0x1ef82df to 0x1efb956, (0x3677 bytes) 2024-09-13T05:03:02.530 INFO:teuthology.orchestra.run.smithi053.stdout:4931 read 0xa328818 thru 0xa32f00a (0x67f3 bytes) 2024-09-13T05:03:02.530 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:02.530 INFO:teuthology.orchestra.run.smithi053.stdout:4933 punch from 0x716779a to 0x71765d2, (0xee38 bytes) 2024-09-13T05:03:02.530 INFO:teuthology.orchestra.run.smithi053.stdout:4934 write 0xe08e338 thru 0xe094272 (0x5f3b bytes) 2024-09-13T05:03:02.533 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:02.538 INFO:teuthology.orchestra.run.smithi053.stdout:4935 writesame 0x1ecdff8 thru 0x1ed924d data_size 0x15(0xb256 bytes) 2024-09-13T05:03:02.553 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:02.552+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:02.575 INFO:teuthology.orchestra.run.smithi053.stdout:4936 read 0x738504b thru 0x738f7cb (0xa781 bytes) 2024-09-13T05:03:02.576 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:02.576 INFO:teuthology.orchestra.run.smithi053.stdout:4939 write 0x336840a thru 0x3375356 (0xcf4d bytes) 2024-09-13T05:03:02.579 INFO:teuthology.orchestra.run.smithi053.stdout:4941 write 0x187cd8e thru 0x188b7b5 (0xea28 bytes) 2024-09-13T05:03:02.591 INFO:teuthology.orchestra.run.smithi053.stdout:4942 write 0x13393ed thru 0x13466a8 (0xd2bc bytes) 2024-09-13T05:03:02.605 INFO:teuthology.orchestra.run.smithi053.stdout:4946 write 0x9738af8 thru 0x9744273 (0xb77c bytes) 2024-09-13T05:03:02.618 INFO:teuthology.orchestra.run.smithi053.stdout:4947 writesame 0xdf1f517 thru 0xdf2313d data_size 0x9(0x3c27 bytes) 2024-09-13T05:03:02.624 INFO:teuthology.orchestra.run.smithi053.stdout:4948 read 0x68e3617 thru 0x68e67c5 (0x31af bytes) 2024-09-13T05:03:02.624 INFO:teuthology.orchestra.run.smithi053.stdout:4949 read 0x8937d6d thru 0x893de8f (0x6123 bytes) 2024-09-13T05:03:02.624 INFO:teuthology.orchestra.run.smithi053.stdout:4950 writesame 0xc31eeff thru 0xc325afa data_size 0x2(0x6bfc bytes) 2024-09-13T05:03:02.633 INFO:teuthology.orchestra.run.smithi053.stdout:4951 punch from 0xe85f593 to 0xe86c51f, (0xcf8c bytes) 2024-09-13T05:03:02.633 INFO:teuthology.orchestra.run.smithi053.stdout:4952 read 0x996cb30 thru 0x997bb57 (0xf028 bytes) 2024-09-13T05:03:02.634 INFO:teuthology.orchestra.run.smithi053.stdout:4954 write 0x291d241 thru 0x292cf26 (0xfce6 bytes) 2024-09-13T05:03:02.647 INFO:teuthology.orchestra.run.smithi053.stdout:4956 read 0x5c0acac thru 0x5c0c800 (0x1b55 bytes) 2024-09-13T05:03:02.647 INFO:teuthology.orchestra.run.smithi053.stdout:4958 read 0x5747cba thru 0x574ece4 (0x702b bytes) 2024-09-13T05:03:02.647 INFO:teuthology.orchestra.run.smithi053.stdout:4959 read 0x337b192 thru 0x338052d (0x539c bytes) 2024-09-13T05:03:02.654 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:02.654+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:02.663 INFO:teuthology.orchestra.run.smithi053.stdout:4961 trunc from 0xe86f827 to 0xc02398 2024-09-13T05:03:02.671 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:02.671 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:02.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:02.726 INFO:teuthology.orchestra.run.smithi053.stdout:4962 read 0x2fd770 thru 0x30ac0c (0xd49d bytes) 2024-09-13T05:03:02.738 INFO:teuthology.orchestra.run.smithi053.stdout:4964 trunc from 0xc02398 to 0xa42d97e 2024-09-13T05:03:02.743 INFO:teuthology.orchestra.run.smithi053.stdout:4965 writesame 0x1ed90c4 thru 0x1edde97 data_size 0x22(0x4dd4 bytes) 2024-09-13T05:03:02.749 INFO:teuthology.orchestra.run.smithi053.stdout:4966 punch from 0x51fd990 to 0x5205adc, (0x814c bytes) 2024-09-13T05:03:02.750 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:02.750 INFO:teuthology.orchestra.run.smithi053.stdout:4969 write 0x1a3401 thru 0x1a5e9b (0x2a9b bytes) 2024-09-13T05:03:02.755 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:02.754+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:02.766 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:02.766 INFO:teuthology.orchestra.run.smithi053.stdout:4971 write 0xa05dab9 thru 0xa06a95b (0xcea3 bytes) 2024-09-13T05:03:02.777 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:02.777 INFO:teuthology.orchestra.run.smithi053.stdout:4973 read 0x841f60b thru 0x8426b43 (0x7539 bytes) 2024-09-13T05:03:02.786 INFO:teuthology.orchestra.run.smithi053.stdout:4974 write 0xcf6fe5f thru 0xcf765e3 (0x6785 bytes) 2024-09-13T05:03:02.795 INFO:teuthology.orchestra.run.smithi053.stdout:4975 punch from 0x8276b98 to 0x8278926, (0x1d8e bytes) 2024-09-13T05:03:02.795 INFO:teuthology.orchestra.run.smithi053.stdout:4978 punch from 0x135b2cb to 0x13637cc, (0x8501 bytes) 2024-09-13T05:03:02.795 INFO:teuthology.orchestra.run.smithi053.stdout:4979 read 0x35cd315 thru 0x35d36f7 (0x63e3 bytes) 2024-09-13T05:03:02.800 INFO:teuthology.orchestra.run.smithi053.stdout:4980 write 0xd4829f4 thru 0xd4895ab (0x6bb8 bytes) 2024-09-13T05:03:02.809 INFO:teuthology.orchestra.run.smithi053.stdout:4981 clone 34 order 25 su 4194304 sc 13 2024-09-13T05:03:02.849 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:02.849 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:02.856 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:02.856+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:02.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:02.905 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:03:02.957 INFO:teuthology.orchestra.run.smithi053.stdout:leaving image image_client.0-clone33 intact 2024-09-13T05:03:02.958 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:02.957+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:03.013 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:03.014 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:03.023 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:03.065 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:03.064+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:03.100 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-13T05:03:03.160 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:03.159+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:03.186 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:03.186 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:03.195 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:03.279 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:03.279+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:03.345 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:03.346 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:03.353 INFO:teuthology.orchestra.run.smithi053.stdout:4982 write 0xbba9c16 thru 0xbbb110b (0x74f6 bytes) 2024-09-13T05:03:03.355 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:03.361 INFO:teuthology.orchestra.run.smithi053.stdout:4983 punch from 0x5c44bd8 to 0x5c4c01b, (0x7443 bytes) 2024-09-13T05:03:03.362 INFO:teuthology.orchestra.run.smithi053.stdout:4984 read 0x45d233a thru 0x45da89b (0x8562 bytes) 2024-09-13T05:03:03.362 INFO:teuthology.orchestra.run.smithi053.stdout:4985 write 0x26984c0 thru 0x26a5a6d (0xd5ae bytes) 2024-09-13T05:03:03.362 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:03.361+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:03.366 INFO:teuthology.orchestra.run.smithi053.stdout:4986 writesame 0xb4a4b8d thru 0xb4a5b98 data_size 0x4(0x100c bytes) 2024-09-13T05:03:03.377 INFO:teuthology.orchestra.run.smithi053.stdout:4987 write 0xed732bc thru 0xed7d671 (0xa3b6 bytes) 2024-09-13T05:03:03.379 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:03.380 INFO:teuthology.orchestra.run.smithi053.stdout:4989 write 0xd7be7ce thru 0xd7c9a93 (0xb2c6 bytes) 2024-09-13T05:03:03.462 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:03.462+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:03.477 INFO:teuthology.orchestra.run.smithi053.stdout:4990 read 0x9b37664 thru 0x9b37aef (0x48c bytes) 2024-09-13T05:03:03.477 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:03.477 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:03.477 INFO:teuthology.orchestra.run.smithi053.stdout:4994 punch from 0xb9e43a4 to 0xb9eb202, (0x6e5e bytes) 2024-09-13T05:03:03.477 INFO:teuthology.orchestra.run.smithi053.stdout:4996 write 0xa84d2ef thru 0xa858838 (0xb54a bytes) 2024-09-13T05:03:03.481 INFO:teuthology.orchestra.run.smithi053.stdout:4997 writesame 0x9ef89b7 thru 0x9f080df data_size 0x17(0xf729 bytes) 2024-09-13T05:03:03.497 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:03.498 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:03.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:03.551 INFO:teuthology.orchestra.run.smithi053.stdout:4998 write 0x32f6a21 thru 0x32fdfa5 (0x7585 bytes) 2024-09-13T05:03:03.555 INFO:teuthology.orchestra.run.smithi053.stdout:4999 read 0x3acd281 thru 0x3acf415 (0x2195 bytes) 2024-09-13T05:03:03.555 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:03.555 INFO:teuthology.orchestra.run.smithi053.stdout:5001 writesame 0xa662be1 thru 0xa666285 data_size 0x1(0x36a5 bytes) 2024-09-13T05:03:03.563 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:03.563+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:03.581 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:03.636 INFO:teuthology.orchestra.run.smithi053.stdout:5002 trunc from 0xed7d672 to 0x243fe2b 2024-09-13T05:03:03.650 INFO:teuthology.orchestra.run.smithi053.stdout:5003 read 0x19f247b thru 0x19fb856 (0x93dc bytes) 2024-09-13T05:03:03.650 INFO:teuthology.orchestra.run.smithi053.stdout:5004 writesame 0x4d11ef thru 0x4dec25 data_size 0x9(0xda37 bytes) 2024-09-13T05:03:03.654 INFO:teuthology.orchestra.run.smithi053.stdout:5005 read 0xb6f209 thru 0xb72f52 (0x3d4a bytes) 2024-09-13T05:03:03.664 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:03.663+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:03.668 INFO:teuthology.orchestra.run.smithi053.stdout:5007 write 0x85f228d thru 0x86014a1 (0xf215 bytes) 2024-09-13T05:03:03.669 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:03.676 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:03.677 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:03.677 INFO:teuthology.orchestra.run.smithi053.stdout:5010 writesame 0x9ea7cac thru 0x9eb39db data_size 0x18(0xbd30 bytes) 2024-09-13T05:03:03.678 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:03.678 INFO:teuthology.orchestra.run.smithi053.stdout:5015 clone 35 order 24 su 65536 sc 3 2024-09-13T05:03:03.686 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:03.717 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:03.717 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:03.734 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:03.735 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:03.750 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:03.751 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:03.765 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:03.765+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:03.766 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:03.766 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:03.784 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:03.784 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:03.796 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:03.796 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:03.796 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:03.800 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:03.800 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:03.805 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:03.817 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:03.818 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:03.834 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:03.834 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:03.890 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:03.885+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:03.930 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:03.930 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.410716280000003, 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.931 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:03.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:03.936 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 49.410544975000001, 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.937 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:03.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:03.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.943 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.4836047319999999, 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:03.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.946 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.4835949529999999, 2024-09-13T05:03:03.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.950 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:03.951 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.952 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.953 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.4835809150000001, 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.954 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.955 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.956 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.957 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.4835547489999996, 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.958 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.959 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:03.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3786762970000002, 2024-09-13T05:03:03.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:03.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:03.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.967 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3764222620000002, 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:03.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3764058879999999, 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.973 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:03.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:03.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3745757970000003, 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.977 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.978 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:03.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3745506729999999, 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:03.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3745402709999999, 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.986 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:03.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.990 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.372221905, 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.991 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:03.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:03.995 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3721973219999999, 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:03.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:03.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:03.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3721143590000002, 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:04.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:04.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.372093778, 2024-09-13T05:03:04.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:04.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.009 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.372077204, 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.010 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:04.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.014 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:04.015 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:04.015 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3720602069999996, 2024-09-13T05:03:04.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:04.015 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:04.015 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:04.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:04.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:04.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.3707635219999998, 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:04.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:04.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 7.370746638, 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:04.024 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:04.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:04.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:04.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:04.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:04.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.029 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:04.029 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:04.029 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:04.031 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:04.031 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone34 from 0x9eb39dc (overlap 0x243fe2b) to 0x121ff15 2024-09-13T05:03:04.031 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:04.029+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:04.050 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-13T05:03:04.137 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:04.135+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:04.169 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:04.169 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:04.169 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:04.169 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:04.181 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:04.268 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:04.267+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:04.325 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:04.326 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:04.326 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:04.326 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:04.326 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:04.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:04.335 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:04.332+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:04.409 INFO:teuthology.orchestra.run.smithi053.stdout:5016 write 0x4707d24 thru 0x4712c6a (0xaf47 bytes) 2024-09-13T05:03:04.434 INFO:teuthology.orchestra.run.smithi053.stdout:5017 punch from 0x6d2556b to 0x6d2c91a, (0x73af bytes) 2024-09-13T05:03:04.434 INFO:teuthology.orchestra.run.smithi053.stdout:5018 read 0xf8cd04 thru 0xf99e02 (0xd0ff bytes) 2024-09-13T05:03:04.434 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:04.433+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:04.434 INFO:teuthology.orchestra.run.smithi053.stdout:5019 writesame 0x5f39224 thru 0x5f394e8 data_size 0x1(0x2c5 bytes) 2024-09-13T05:03:04.443 INFO:teuthology.orchestra.run.smithi053.stdout:5020 write 0xb1cdde2 thru 0xb1d04ec (0x270b bytes) 2024-09-13T05:03:04.451 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.451 INFO:teuthology.orchestra.run.smithi053.stdout:5025 writesame 0x8f52d1e thru 0x8f5c6e6 data_size 0x3(0x99c9 bytes) 2024-09-13T05:03:04.468 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.469 INFO:teuthology.orchestra.run.smithi053.stdout:5029 read 0x275f782 thru 0x276a5f1 (0xae70 bytes) 2024-09-13T05:03:04.469 INFO:teuthology.orchestra.run.smithi053.stdout:5031 punch from 0x4ff5417 to 0x4ffbb82, (0x676b bytes) 2024-09-13T05:03:04.469 INFO:teuthology.orchestra.run.smithi053.stdout:5033 write 0x34405ef thru 0x3440dba (0x7cc bytes) 2024-09-13T05:03:04.470 INFO:teuthology.orchestra.run.smithi053.stdout:5034 write 0xa414370 thru 0xa41e2ad (0x9f3e bytes) 2024-09-13T05:03:04.477 INFO:teuthology.orchestra.run.smithi053.stdout:5035 write 0xbdeaab5 thru 0xbdef5bf (0x4b0b bytes) 2024-09-13T05:03:04.482 INFO:teuthology.orchestra.run.smithi053.stdout:5036 writesame 0xba92f2c thru 0xbaa100f data_size 0x4a(0xe0e4 bytes) 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:5037 punch from 0xa82dbe8 to 0xa834331, (0x6749 bytes) 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:5040 punch from 0x1d10f7e to 0x1d1d9f4, (0xca76 bytes) 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:5043 punch from 0x3d0358b to 0x3d099f3, (0x6468 bytes) 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:5044 read 0x3b3df5 thru 0x3b8d75 (0x4f81 bytes) 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:5045 read 0x84f4bcd thru 0x85036b5 (0xeae9 bytes) 2024-09-13T05:03:04.485 INFO:teuthology.orchestra.run.smithi053.stdout:5047 write 0xf9fb30 thru 0xfaee5e (0xf32f bytes) 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.495 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250428202000002, 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.496 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:04.497 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.501 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.502 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.503 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.504 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250411947000003, 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.505 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:04.506 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.507 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.507 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.507 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:04.507 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.507 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.508 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:03:04.509 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:03:04.509 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.509 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.509 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.509 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:03:04.510 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:03:04.510 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.510 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.510 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.510 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:03:04.511 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:03:04.511 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.511 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.511 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.512 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250394858, 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.513 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.514 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:03:04.515 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.516 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.517 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250377991000001, 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.518 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:03:04.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250328693, 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:03:04.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:04.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250305423999997, 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:03:04.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250289690000002, 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:04.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:03:04.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.537 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.538 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250266709999998, 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:03:04.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:03:04.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250217644000003, 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:03:04.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:04.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:04.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:03:04.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:03:04.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:03:04.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:03:04.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250096329000002, 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.551 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:03:04.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.559 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250041115000002, 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:03:04.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:03:04.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:03:04.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:03:04.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:03:04.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.568 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.568 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:04.568 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250020825999997, 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.569 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:04.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:03:04.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:03:04.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.577 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.577 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.250003645, 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:03:04.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.582 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249916134999999, 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:03:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:03:04.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:03:04.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:03:04.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249896886999998, 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:03:04.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249835771999997, 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.594 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249818597000001, 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:03:04.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:03:04.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249802187, 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:03:04.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249781202999998, 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:03:04.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 37.249763778999998, 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:04.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:04.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:04.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:04.616 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:04.619 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:04.619 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:04.617+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5050 read 0xd08013 thru 0xd0f644 (0x7632 bytes) 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5051 writesame 0xdf0fd8a thru 0xdf142cb data_size 0x2d(0x4542 bytes) 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5053 trunc from 0xdf142cc to 0x91fed22 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5054 read 0x7504050 thru 0x7506b67 (0x2b18 bytes) 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5055 read 0x26ceb7e thru 0x26d749b (0x891e bytes) 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5056 writesame 0x3e28ebd thru 0x3e36682 data_size 0x47(0xd7c6 bytes) 2024-09-13T05:03:04.621 INFO:teuthology.orchestra.run.smithi053.stdout:5057 write 0x5b400a8 thru 0x5b47f03 (0x7e5c bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5059 writesame 0x782a60a thru 0x78323e8 data_size 0x17(0x7ddf bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5060 punch from 0x4bbaf25 to 0x4bc9174, (0xe24f bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5061 read 0x8b298de thru 0x8b329e9 (0x910c bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5062 read 0x3996e08 thru 0x3997074 (0x26d bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5063 writesame 0x3aa6a52 thru 0x3ab44a5 data_size 0x59(0xda54 bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5064 write 0x918f1e8 thru 0x919e2f0 (0xf109 bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5065 read 0x482427d thru 0x482ddb4 (0x9b38 bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5066 writesame 0xbdbde9a thru 0xbdc7853 data_size 0x15(0x99ba bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5068 punch from 0x2ac6589 to 0x2ad30cf, (0xcb46 bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5069 writesame 0xc96f78d thru 0xc97c34f data_size 0x1(0xcbc3 bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5071 punch from 0x8b2f678 to 0x8b306a3, (0x102b bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5074 punch from 0xb9053bd to 0xb912901, (0xd544 bytes) 2024-09-13T05:03:04.622 INFO:teuthology.orchestra.run.smithi053.stdout:5075 write 0x79ed295 thru 0x79f3f61 (0x6ccd bytes) 2024-09-13T05:03:04.623 INFO:teuthology.orchestra.run.smithi053.stdout:5076 read 0x14d1cef thru 0x14d5e4b (0x415d bytes) 2024-09-13T05:03:04.623 INFO:teuthology.orchestra.run.smithi053.stdout:5077 write 0x977e2a3 thru 0x9783031 (0x4d8f bytes) 2024-09-13T05:03:04.623 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.623 INFO:teuthology.orchestra.run.smithi053.stdout:5079 read 0x51d08ea thru 0x51d2cce (0x23e5 bytes) 2024-09-13T05:03:04.623 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.623 INFO:teuthology.orchestra.run.smithi053.stdout:5083 trunc from 0xc97c350 to 0x5279973 2024-09-13T05:03:04.624 INFO:teuthology.orchestra.run.smithi053.stdout:5084 read 0x173a48c thru 0x173ab77 (0x6ec bytes) 2024-09-13T05:03:04.635 INFO:teuthology.orchestra.run.smithi053.stdout:5087 write 0x97d1fc2 thru 0x97db7ee (0x982d bytes) 2024-09-13T05:03:04.638 INFO:teuthology.orchestra.run.smithi053.stdout:5090 read 0x25fd26d thru 0x25fe64f (0x13e3 bytes) 2024-09-13T05:03:04.640 INFO:teuthology.orchestra.run.smithi053.stdout:5091 trunc from 0x97db7ef to 0xae6f2bc 2024-09-13T05:03:04.645 INFO:teuthology.orchestra.run.smithi053.stdout:5092 punch from 0x84641cb to 0x846875f, (0x4594 bytes) 2024-09-13T05:03:04.651 INFO:teuthology.orchestra.run.smithi053.stdout:5093 write 0xb70be4b thru 0xb713dc3 (0x7f79 bytes) 2024-09-13T05:03:04.656 INFO:teuthology.orchestra.run.smithi053.stdout:5094 punch from 0x2fb7a29 to 0x2fc7921, (0xfef8 bytes) 2024-09-13T05:03:04.656 INFO:teuthology.orchestra.run.smithi053.stdout:5095 punch from 0x73187da to 0x7319b7c, (0x13a2 bytes) 2024-09-13T05:03:04.656 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.663 INFO:teuthology.orchestra.run.smithi053.stdout:5097 write 0xcc7a9c6 thru 0xcc7e53c (0x3b77 bytes) 2024-09-13T05:03:04.666 INFO:teuthology.orchestra.run.smithi053.stdout:5100 punch from 0x92bf8ab to 0x92c2403, (0x2b58 bytes) 2024-09-13T05:03:04.666 INFO:teuthology.orchestra.run.smithi053.stdout:5101 write 0xa5cdf4d thru 0xa5d372e (0x57e2 bytes) 2024-09-13T05:03:04.674 INFO:teuthology.orchestra.run.smithi053.stdout:5102 write 0xe4d809a thru 0xe4d8329 (0x290 bytes) 2024-09-13T05:03:04.676 INFO:teuthology.orchestra.run.smithi053.stdout:5103 punch from 0x441441b to 0x4423b99, (0xf77e bytes) 2024-09-13T05:03:04.676 INFO:teuthology.orchestra.run.smithi053.stdout:5104 write 0xd1d3702 thru 0xd1d5442 (0x1d41 bytes) 2024-09-13T05:03:04.692 INFO:teuthology.orchestra.run.smithi053.stdout:5105 trunc from 0xe4d832a to 0x33a42f4 2024-09-13T05:03:04.711 INFO:teuthology.orchestra.run.smithi053.stdout:5106 write 0x5b38c1d thru 0x5b46644 (0xda28 bytes) 2024-09-13T05:03:04.713 INFO:teuthology.orchestra.run.smithi053.stdout:5107 write 0x1af72b3 thru 0x1b012ac (0x9ffa bytes) 2024-09-13T05:03:04.718 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:04.717+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:04.742 INFO:teuthology.orchestra.run.smithi053.stdout:5109 write 0x4b31f3f thru 0x4b3dac8 (0xbb8a bytes) 2024-09-13T05:03:04.750 INFO:teuthology.orchestra.run.smithi053.stdout:5111 trunc from 0x5b46645 to 0xad826c1 2024-09-13T05:03:04.755 INFO:teuthology.orchestra.run.smithi053.stdout:5113 read 0x264c6b0 thru 0x2652cf8 (0x6649 bytes) 2024-09-13T05:03:04.755 INFO:teuthology.orchestra.run.smithi053.stdout:5114 write 0x87f136 thru 0x8895fb (0xa4c6 bytes) 2024-09-13T05:03:04.758 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.758 INFO:teuthology.orchestra.run.smithi053.stdout:5116 read 0x69466fb thru 0x69541ff (0xdb05 bytes) 2024-09-13T05:03:04.758 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.758 INFO:teuthology.orchestra.run.smithi053.stdout:5123 read 0x7ea586c thru 0x7eadab9 (0x824e bytes) 2024-09-13T05:03:04.758 INFO:teuthology.orchestra.run.smithi053.stdout:5124 write 0x2f29c8 thru 0x2f8a9d (0x60d6 bytes) 2024-09-13T05:03:04.759 INFO:teuthology.orchestra.run.smithi053.stdout:5126 writesame 0xa3736a3 thru 0xa378bca data_size 0x28(0x5528 bytes) 2024-09-13T05:03:04.762 INFO:teuthology.orchestra.run.smithi053.stdout:5127 read 0x5bee6a8 thru 0x5bf1233 (0x2b8c bytes) 2024-09-13T05:03:04.762 INFO:teuthology.orchestra.run.smithi053.stdout:5128 write 0x3ba6121 thru 0x3baf011 (0x8ef1 bytes) 2024-09-13T05:03:04.764 INFO:teuthology.orchestra.run.smithi053.stdout:5129 write 0x4c000a0 thru 0x4c0939b (0x92fc bytes) 2024-09-13T05:03:04.766 INFO:teuthology.orchestra.run.smithi053.stdout:5130 punch from 0x361359e to 0x3613e9d, (0x8ff bytes) 2024-09-13T05:03:04.766 INFO:teuthology.orchestra.run.smithi053.stdout:5131 write 0x6bf25a4 thru 0x6bf88af (0x630c bytes) 2024-09-13T05:03:04.774 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:04.774 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:04.774 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:04.774 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:04.779 INFO:teuthology.orchestra.run.smithi053.stdout:5132 writesame 0xe65a01c thru 0xe6648a5 data_size 0x8d(0xa88a bytes) 2024-09-13T05:03:04.779 INFO:teuthology.orchestra.run.smithi053.stdout:5133 read 0xa6b0062 thru 0xa6b48a8 (0x4847 bytes) 2024-09-13T05:03:04.779 INFO:teuthology.orchestra.run.smithi053.stdout:5134 write 0xcee7aa5 thru 0xcee8b15 (0x1071 bytes) 2024-09-13T05:03:04.781 INFO:teuthology.orchestra.run.smithi053.stdout:5135 read 0x46a93ae thru 0x46b660e (0xd261 bytes) 2024-09-13T05:03:04.782 INFO:teuthology.orchestra.run.smithi053.stdout:5136 write 0x5ae95e6 thru 0x5aeff36 (0x6951 bytes) 2024-09-13T05:03:04.783 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:04.783 INFO:teuthology.orchestra.run.smithi053.stdout:5137 read 0x1050aa7 thru 0x1060823 (0xfd7d bytes) 2024-09-13T05:03:04.784 INFO:teuthology.orchestra.run.smithi053.stdout:5138 write 0x95808c5 thru 0x958571d (0x4e59 bytes) 2024-09-13T05:03:04.787 INFO:teuthology.orchestra.run.smithi053.stdout:5140 punch from 0x42921f8 to 0x429f39e, (0xd1a6 bytes) 2024-09-13T05:03:04.787 INFO:teuthology.orchestra.run.smithi053.stdout:5144 write 0xc47cde1 thru 0xc4824ce (0x56ee bytes) 2024-09-13T05:03:04.791 INFO:teuthology.orchestra.run.smithi053.stdout:5145 read 0x7f9ef5e thru 0x7fa4b7e (0x5c21 bytes) 2024-09-13T05:03:04.791 INFO:teuthology.orchestra.run.smithi053.stdout:5146 read 0x1170757 thru 0x11775d7 (0x6e81 bytes) 2024-09-13T05:03:04.792 INFO:teuthology.orchestra.run.smithi053.stdout:5147 read 0x7980500 thru 0x798e6f2 (0xe1f3 bytes) 2024-09-13T05:03:04.792 INFO:teuthology.orchestra.run.smithi053.stdout:5148 write 0x4044a9b thru 0x4045aa5 (0x100b bytes) 2024-09-13T05:03:04.804 INFO:teuthology.orchestra.run.smithi053.stdout:5152 trunc from 0xe6648a6 to 0x49260fd 2024-09-13T05:03:04.815 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.816 INFO:teuthology.orchestra.run.smithi053.stdout:5155 read 0x21d0275 thru 0x21d05e6 (0x372 bytes) 2024-09-13T05:03:04.819 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:04.818+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:04.830 INFO:teuthology.orchestra.run.smithi053.stdout:5157 write 0xca128d5 thru 0xca19d49 (0x7475 bytes) 2024-09-13T05:03:04.833 INFO:teuthology.orchestra.run.smithi053.stdout:5158 read 0xc8dbdea thru 0xc8ddc15 (0x1e2c bytes) 2024-09-13T05:03:04.841 INFO:teuthology.orchestra.run.smithi053.stdout:5159 trunc from 0xca19d4a to 0x59867ce 2024-09-13T05:03:04.862 INFO:teuthology.orchestra.run.smithi053.stdout:5160 writesame 0x82a5eb3 thru 0x82ad18e data_size 0x2(0x72dc bytes) 2024-09-13T05:03:04.862 INFO:teuthology.orchestra.run.smithi053.stdout:5161 read 0x48706d8 thru 0x48748cf (0x41f8 bytes) 2024-09-13T05:03:04.871 INFO:teuthology.orchestra.run.smithi053.stdout:5162 trunc from 0x82ad18f to 0x467bb2 2024-09-13T05:03:04.884 INFO:teuthology.orchestra.run.smithi053.stdout:5164 read 0x22cd77 thru 0x238a98 (0xbd22 bytes) 2024-09-13T05:03:04.885 INFO:teuthology.orchestra.run.smithi053.stdout:5166 read 0x2e0788 thru 0x2e5b93 (0x540c bytes) 2024-09-13T05:03:04.886 INFO:teuthology.orchestra.run.smithi053.stdout:5167 write 0x435d41 thru 0x43f270 (0x9530 bytes) 2024-09-13T05:03:04.904 INFO:teuthology.orchestra.run.smithi053.stdout:5170 trunc from 0x467bb2 to 0xecbe034 2024-09-13T05:03:04.909 INFO:teuthology.orchestra.run.smithi053.stdout:5171 punch from 0xebce5ea to 0xebd6747, (0x815d bytes) 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:5172 read 0x5cf55f9 thru 0x5cfff01 (0xa909 bytes) 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:5175 read 0x6897b50 thru 0x68a2235 (0xa6e6 bytes) 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:5177 read 0x29826ee thru 0x298412e (0x1a41 bytes) 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:5179 read 0xb94bf56 thru 0xb94eb18 (0x2bc3 bytes) 2024-09-13T05:03:04.910 INFO:teuthology.orchestra.run.smithi053.stdout:5180 writesame 0x76fa8ae thru 0x76ffd33 data_size 0x1f(0x5486 bytes) 2024-09-13T05:03:04.919 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:04.919+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:04.920 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:03:04.927 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:04.928 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:04.928 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:04.928 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:04.928 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:04.929 INFO:teuthology.orchestra.run.smithi053.stdout:5183 trunc from 0xecbe034 to 0x7db25a 2024-09-13T05:03:04.937 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:04.958 INFO:teuthology.orchestra.run.smithi053.stdout:5185 write 0xa7159a6 thru 0xa71e67c (0x8cd7 bytes) 2024-09-13T05:03:04.962 INFO:teuthology.orchestra.run.smithi053.stdout:5186 punch from 0x3f3a471 to 0x3f42ca8, (0x8837 bytes) 2024-09-13T05:03:04.962 INFO:teuthology.orchestra.run.smithi053.stdout:5187 write 0x45f9c3c thru 0x4604bbc (0xaf81 bytes) 2024-09-13T05:03:04.966 INFO:teuthology.orchestra.run.smithi053.stdout:5188 read 0xb85211 thru 0xb87787 (0x2577 bytes) 2024-09-13T05:03:04.973 INFO:teuthology.orchestra.run.smithi053.stdout:5191 write 0xa7cf0b4 thru 0xa7db43c (0xc389 bytes) 2024-09-13T05:03:04.988 INFO:teuthology.orchestra.run.smithi053.stdout:5192 trunc from 0xa7db43d to 0x14d9b7 2024-09-13T05:03:05.012 INFO:teuthology.orchestra.run.smithi053.stdout:5195 write 0x4c45576 thru 0x4c526e5 (0xd170 bytes) 2024-09-13T05:03:05.015 INFO:teuthology.orchestra.run.smithi053.stdout:5196 write 0x486f94 thru 0x488964 (0x19d1 bytes) 2024-09-13T05:03:05.019 INFO:teuthology.orchestra.run.smithi053.stdout:5197 punch from 0x29a453f to 0x29af8da, (0xb39b bytes) 2024-09-13T05:03:05.019 INFO:teuthology.orchestra.run.smithi053.stdout:5198 writesame 0x23c283a thru 0x23c368a data_size 0x1(0xe51 bytes) 2024-09-13T05:03:05.020 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:05.019+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:05.021 INFO:teuthology.orchestra.run.smithi053.stdout:5199 trunc from 0x4c526e6 to 0x283a0c8 2024-09-13T05:03:05.033 INFO:teuthology.orchestra.run.smithi053.stdout:5200 punch from 0xd5eef4 to 0xd641c0, (0x52cc bytes) 2024-09-13T05:03:05.053 INFO:teuthology.orchestra.run.smithi053.stdout:5204 writesame 0xe2ec403 thru 0xe2f36c1 data_size 0x2f(0x72bf bytes) 2024-09-13T05:03:05.055 INFO:teuthology.orchestra.run.smithi053.stdout:5206 trunc from 0xe2f36c2 to 0xbeba6ab 2024-09-13T05:03:05.061 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:05.061 INFO:teuthology.orchestra.run.smithi053.stdout:5208 write 0x28ad2d2 thru 0x28b6ca9 (0x99d8 bytes) 2024-09-13T05:03:05.063 INFO:teuthology.orchestra.run.smithi053.stdout:5209 punch from 0x66744d4 to 0x6674656, (0x182 bytes) 2024-09-13T05:03:05.069 INFO:teuthology.orchestra.run.smithi053.stdout:5210 write 0xd5b9687 thru 0xd5bc397 (0x2d11 bytes) 2024-09-13T05:03:05.073 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:05.073 INFO:teuthology.orchestra.run.smithi053.stdout:5212 write 0x661b082 thru 0x662745d (0xc3dc bytes) 2024-09-13T05:03:05.077 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:05.077 INFO:teuthology.orchestra.run.smithi053.stdout:5216 punch from 0x97a70cd to 0x97a969f, (0x25d2 bytes) 2024-09-13T05:03:05.081 INFO:teuthology.orchestra.run.smithi053.stdout:5219 trunc from 0xd5bc398 to 0xa3251b2 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.085 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466824057, 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:05.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466805477999999, 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.090 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:05.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.094 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466780684, 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:05.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:05.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:05.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466761328, 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:05.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:05.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466742642, 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:05.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:05.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466717573, 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.110 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:05.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466695198, 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.116 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:05.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466674380000001, 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.121 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:05.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466649075999999, 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:05.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466630350999999, 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:05.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.134 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.46661229, 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:05.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466580059, 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:05.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466560898999999, 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:05.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:05.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:05.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:05.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:05.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466543797, 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:05.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:05.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:05.154 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.4665175, 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:05.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:05.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466491653, 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.160 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:05.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.165 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.165 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:05.165 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466474801, 2024-09-13T05:03:05.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:05.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.167 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:05.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:05.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:05.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:05.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:05.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:05.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:05.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:05.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.174 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466458441, 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:05.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:05.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466442317, 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.179 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:05.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.466421087000001, 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:05.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.188 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:05.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:05.191 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:05.189+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:05.192 INFO:teuthology.orchestra.run.smithi053.stdout:5221 punch from 0x17d57a1 to 0x17e3927, (0xe186 bytes) 2024-09-13T05:03:05.192 INFO:teuthology.orchestra.run.smithi053.stdout:5222 read 0x10a315d thru 0x10aff5a (0xcdfe bytes) 2024-09-13T05:03:05.192 INFO:teuthology.orchestra.run.smithi053.stdout:5223 writesame 0xb2ac924 thru 0xb2af269 data_size 0x9(0x2946 bytes) 2024-09-13T05:03:05.192 INFO:teuthology.orchestra.run.smithi053.stdout:5224 read 0x3d149f2 thru 0x3d23198 (0xe7a7 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5226 write 0x417b14f thru 0x417cdf0 (0x1ca2 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5227 punch from 0x1326e94 to 0x13272b8, (0x424 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5228 read 0x54f0c02 thru 0x54fb13b (0xa53a bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5229 write 0x132940f thru 0x132db19 (0x470b bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5230 writesame 0x21e2c5f thru 0x21f2a1c data_size 0x1(0xfdbe bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5232 write 0x6c0c6b0 thru 0x6c146a0 (0x7ff1 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5233 write 0x9e335ae thru 0x9e37392 (0x3de5 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5234 read 0x71e2758 thru 0x71e53ad (0x2c56 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5237 trunc from 0xb2af26a to 0xc12ad3 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5239 trunc from 0xc12ad3 to 0x4a1c5a6 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5240 punch from 0xd408c4 to 0xd476a7, (0x6de3 bytes) 2024-09-13T05:03:05.193 INFO:teuthology.orchestra.run.smithi053.stdout:5243 read 0x473a6b5 thru 0x474101b (0x6967 bytes) 2024-09-13T05:03:05.194 INFO:teuthology.orchestra.run.smithi053.stdout:5244 trunc from 0x4a1c5a6 to 0x8abc291 2024-09-13T05:03:05.194 INFO:teuthology.orchestra.run.smithi053.stdout:5248 clone 36 order 24 su 1048576 sc 5 2024-09-13T05:03:05.290 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:05.290+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:05.333 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:05.333 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:05.334 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:05.334 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:05.341 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:05.391 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:05.391+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:05.443 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:05.443 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:05.443 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:05.443 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:05.444 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:05.452 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:05.495 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:05.495+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70864 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.974308324000001, 2024-09-13T05:03:05.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038688958000000002, 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70864 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579270+0000", 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.491e-06 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579283+0000", 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2653999999999999e-05 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579289+0000", 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.614870+0000", 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035581387999999999 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615507+0000", 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00063753200000000001 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617957+0000", 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024498380000000002 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.559 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70866 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2596864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.974117965, 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036734241000000001, 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.560 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70866 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579458+0000", 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579462+0000", 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579471+0000", 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9639999999999992e-06 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579477+0000", 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0000000000000002e-06 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615818+0000", 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036340823000000001 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616117+0000", 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029901500000000002 2024-09-13T05:03:05.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616193+0000", 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5829999999999997e-05 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70869 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.974093728, 2024-09-13T05:03:05.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036867536999999999, 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70869 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579482+0000", 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579485+0000", 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3329999999999999e-06 2024-09-13T05:03:05.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579491+0000", 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1580000000000002e-06 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579496+0000", 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8679999999999998e-06 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616223+0000", 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036727187000000001 2024-09-13T05:03:05.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616290+0000", 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7160000000000001e-05 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616350+0000", 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0248999999999999e-05 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70877 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.973537405, 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036468836999999997, 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70877 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:05.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580043+0000", 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3989999999999997e-06 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580052+0000", 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6319999999999997e-06 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580065+0000", 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3288000000000001e-05 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616372+0000", 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036307147999999997 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616447+0000", 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5296999999999994e-05 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616508+0000", 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0751999999999997e-05 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70879 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1130496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.972790563, 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035893590000000003, 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.573 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70879 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580785+0000", 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580792+0000", 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9029999999999998e-06 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580804+0000", 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1545999999999999e-05 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580811+0000", 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2609999999999996e-06 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616530+0000", 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035719133 2024-09-13T05:03:05.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616604+0000", 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3712000000000002e-05 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616680+0000", 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5957000000000005e-05 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70882 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.972759097000001, 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036032559999999998, 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70882 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580817+0000", 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580820+0000", 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5720000000000001e-06 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580826+0000", 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4130000000000003e-06 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580831+0000", 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.865e-06 2024-09-13T05:03:05.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616703+0000", 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035871694000000003 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616782+0000", 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9369999999999994e-05 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616850+0000", 2024-09-13T05:03:05.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8118000000000001e-05 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70891 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1089536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.972740004, 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036169077000000001, 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.582 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70891 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580836+0000", 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580839+0000", 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3760000000000002e-06 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580853+0000", 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4735999999999999e-05 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580859+0000", 2024-09-13T05:03:05.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3329999999999996e-06 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616872+0000", 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036013495999999999 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616945+0000", 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2646999999999997e-05 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617006+0000", 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0853000000000002e-05 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70893 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.972706178999999, 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036286807999999997, 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70893 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580873+0000", 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7930000000000002e-06 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580880+0000", 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9970000000000001e-06 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580885+0000", 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5350000000000004e-06 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036141860999999997 2024-09-13T05:03:05.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617091+0000", 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3730000000000001e-05 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6412e-05 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70895 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1187840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.972595427, 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036346900000000001, 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70895 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.592 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:05.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580985+0000", 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4009999999999999e-06 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581000+0000", 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5024999999999999e-05 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581006+0000", 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3080000000000004e-06 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617178+0000", 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036171571999999999 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617328+0000", 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1537000000000002e-05 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71006 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.967993113, 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035821812000000001, 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.596 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71006 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:05.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585583+0000", 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585586+0000", 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585593+0000", 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9210000000000003e-06 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585598+0000", 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0159999999999999e-06 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621279+0000", 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035681069000000003 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621346+0000", 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7080000000000001e-05 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621405+0000", 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9401999999999998e-05 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71008 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.967971852, 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035951295000000001, 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71008 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585604+0000", 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585607+0000", 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.266e-06 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585613+0000", 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585622+0000", 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7590000000000003e-06 2024-09-13T05:03:05.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621431+0000", 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035809331 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621495+0000", 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4611999999999997e-05 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621556+0000", 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0591999999999999e-05 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71009 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.967950451, 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036076001000000003, 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.605 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71009 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585634+0000", 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.055e-06 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585640+0000", 2024-09-13T05:03:05.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4189999999999999e-06 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585646+0000", 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621580+0000", 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035934440999999998 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621644+0000", 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3927e-05 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621702+0000", 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7855000000000001e-05 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71011 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.609 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.967921302000001, 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036193110000000001, 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71011 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.610 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585658+0000", 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7329999999999999e-06 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585666+0000", 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7280000000000001e-06 2024-09-13T05:03:05.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585671+0000", 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-06 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621726+0000", 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036054614999999998 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621790+0000", 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3868000000000006e-05 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621848+0000", 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8913000000000003e-05 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71071 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.614 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.967012186, 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035743500999999997, 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71071 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:05.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586566+0000", 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3209999999999999e-06 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586572+0000", 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586578+0000", 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622184+0000", 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035606024999999999 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622248+0000", 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4691999999999997e-05 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622308+0000", 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9663000000000001e-05 2024-09-13T05:03:05.618 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71096 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.966037397999999, 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035817952, 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.619 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71096 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:05.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587541+0000", 2024-09-13T05:03:05.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587548+0000", 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4679999999999997e-06 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587561+0000", 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2713e-05 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623235+0000", 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035674375000000001 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623300+0000", 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4505999999999995e-05 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623357+0000", 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7661999999999998e-05 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.623 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71097 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.966008331999999, 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035944619999999997, 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.624 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71097 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587575+0000", 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587582+0000", 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1160000000000001e-06 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587587+0000", 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0490000000000003e-06 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623383+0000", 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035795897 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623448+0000", 2024-09-13T05:03:05.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4604999999999996e-05 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623513+0000", 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5168999999999999e-05 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71115 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.965180723, 2024-09-13T05:03:05.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036023959000000001, 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71115 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588395+0000", 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588400+0000", 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588410+0000", 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0076e-05 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588417+0000", 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6549999999999998e-06 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623694+0000", 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035277159000000002 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623760+0000", 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5968000000000003e-05 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624420+0000", 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066017099999999998 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.632 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71117 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.96493233, 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035938036, 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71117 2024-09-13T05:03:05.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588643+0000", 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588649+0000", 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5210000000000002e-06 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588660+0000", 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1459e-05 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588668+0000", 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5669999999999997e-06 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624447+0000", 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035778841999999998 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624516+0000", 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8774000000000003e-05 2024-09-13T05:03:05.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624582+0000", 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6818999999999998e-05 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71119 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.964613162999999, 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035764241000000002, 2024-09-13T05:03:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71119 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588962+0000", 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588968+0000", 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5770000000000001e-06 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588980+0000", 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1718e-05 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588987+0000", 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624607+0000", 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035619545000000002 2024-09-13T05:03:05.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624670+0000", 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3664999999999994e-05 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624728+0000", 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7433999999999999e-05 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71120 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1220608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 10.964432428, 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035736393999999998, 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71120 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:05.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589143+0000", 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:05.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589149+0000", 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1329999999999998e-06 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589167+0000", 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8236999999999998e-05 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:05.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589174+0000", 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4889999999999998e-06 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624752+0000", 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035577723999999998 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624813+0000", 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1138999999999999e-05 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:05.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624881+0000", 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7489000000000004e-05 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:05.646 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:05.648 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:05.648 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:05.647+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:05.748 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:05.748+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:05.790 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:05.790 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:05.790 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:05.790 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:05.798 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:05.845 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:05.883 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:05.882+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:05.915 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:03:05.943 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:05.943 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:05.943 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:05.943 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:05.943 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:05.950 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:05.950+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:05.952 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:05.990 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone35 from 0x8abc291 (overlap 0x14d9b7) to 0x12c3f1 2024-09-13T05:03:05.992 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:05.992 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:06.013 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:06.013 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:06.029 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:06.029 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:06.045 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:06.046 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:06.051 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:06.050+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:06.057 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-13T05:03:06.063 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:06.063 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:06.080 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:06.080 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:06.092 INFO:teuthology.orchestra.run.smithi053.stdout:5249 write 0x22be6d9 thru 0x22c8277 (0x9b9f bytes) 2024-09-13T05:03:06.097 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:06.097 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:06.102 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:06.102 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:06.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:06.102 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:06.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.102 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.871636027999998, 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.870707918999997, 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.107 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:06.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.869609509999997, 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:06.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:06.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.863549548999998, 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:06.116 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:06.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:06.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:06.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.863540553999997, 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:06.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:06.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:06.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:06.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.863532182999997, 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.127 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.131 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.863522162000002, 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:06.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:06.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:06.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:06.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.863506438999998, 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:06.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.861604270000001, 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:06.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.861596311, 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:06.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.861588623000003, 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:06.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.861577930999999, 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.157 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.161 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.861569690000003, 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:06.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:06.165 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.861546029000003, 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.166 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:06.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:06.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.85990357, 2024-09-13T05:03:06.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:06.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.174 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.859895336999998, 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:06.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:06.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.859672607999997, 2024-09-13T05:03:06.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:06.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:06.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.859660058999999, 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:06.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:06.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.857620832999999, 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.188 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:06.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:06.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.193 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 41.857611054000003, 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:06.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:06.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:06.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:06.199 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:06.200 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:06.200 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:06.201 INFO:teuthology.orchestra.run.smithi053.stdout:5250 write 0x9c9d32 thru 0x9d4ab2 (0xad81 bytes) 2024-09-13T05:03:06.201 INFO:teuthology.orchestra.run.smithi053.stdout:5251 write 0x2fb04fb thru 0x2fb6b03 (0x6609 bytes) 2024-09-13T05:03:06.202 INFO:teuthology.orchestra.run.smithi053.stdout:5252 read 0x5a57fb5 thru 0x5a5b5a5 (0x35f1 bytes) 2024-09-13T05:03:06.202 INFO:teuthology.orchestra.run.smithi053.stdout:5255 read 0x5496a8c thru 0x549cbf8 (0x616d bytes) 2024-09-13T05:03:06.202 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:06.202 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:06.203 INFO:teuthology.orchestra.run.smithi053.stdout:5256 trunc from 0x8abc291 to 0x15c55 2024-09-13T05:03:06.203 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:06.205 INFO:teuthology.orchestra.run.smithi053.stdout:5258 write 0xa58713b thru 0xa58a9fa (0x38c0 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5259 writesame 0x447c106 thru 0x4481734 data_size 0x1(0x562f bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5260 punch from 0x3797efc to 0x37a4c19, (0xcd1d bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5262 write 0xdb62267 thru 0xdb70b05 (0xe89f bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5263 write 0xa860a3b thru 0xa862d48 (0x230e bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5264 read 0x1eb136c thru 0x1eb36c1 (0x2356 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5267 write 0xac52ccd thru 0xac601d4 (0xd508 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5269 read 0x717a73b thru 0x717fcb7 (0x557d bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5270 read 0xb2c0593 thru 0xb2c3578 (0x2fe6 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5271 punch from 0xa6029c6 to 0xa604745, (0x1d7f bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5272 read 0x7477d49 thru 0x7480065 (0x831d bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5273 writesame 0x90d38a4 thru 0x90db7c3 data_size 0x12(0x7f20 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5276 read 0x6f7fb11 thru 0x6f89667 (0x9b57 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5277 write 0xb112393 thru 0xb11aa9a (0x8708 bytes) 2024-09-13T05:03:06.206 INFO:teuthology.orchestra.run.smithi053.stdout:5278 writesame 0xd6254db thru 0xd62d9a8 data_size 0x2(0x84ce bytes) 2024-09-13T05:03:06.207 INFO:teuthology.orchestra.run.smithi053.stdout:5279 write 0x371396f thru 0x371c7b5 (0x8e47 bytes) 2024-09-13T05:03:06.207 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.207 INFO:teuthology.orchestra.run.smithi053.stdout:5281 read 0x154ab1f thru 0x154e4dc (0x39be bytes) 2024-09-13T05:03:06.207 INFO:teuthology.orchestra.run.smithi053.stdout:5282 trunc from 0xdb70b06 to 0xa32c328 2024-09-13T05:03:06.207 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:06.200+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:06.210 INFO:teuthology.orchestra.run.smithi053.stdout:5283 punch from 0x1939f4b to 0x1940d20, (0x6dd5 bytes) 2024-09-13T05:03:06.210 INFO:teuthology.orchestra.run.smithi053.stdout:5284 punch from 0x7ff9085 to 0x8001558, (0x84d3 bytes) 2024-09-13T05:03:06.212 INFO:teuthology.orchestra.run.smithi053.stdout:5285 trunc from 0xa32c328 to 0xbdcdfbe 2024-09-13T05:03:06.217 INFO:teuthology.orchestra.run.smithi053.stdout:5286 read 0x3867aac thru 0x3870b69 (0x90be bytes) 2024-09-13T05:03:06.217 INFO:teuthology.orchestra.run.smithi053.stdout:5287 writesame 0xb7b2de1 thru 0xb7b4ace data_size 0x2e(0x1cee bytes) 2024-09-13T05:03:06.219 INFO:teuthology.orchestra.run.smithi053.stdout:5288 read 0x2623512 thru 0x2626b78 (0x3667 bytes) 2024-09-13T05:03:06.219 INFO:teuthology.orchestra.run.smithi053.stdout:5289 punch from 0x81fcda8 to 0x8205e99, (0x90f1 bytes) 2024-09-13T05:03:06.220 INFO:teuthology.orchestra.run.smithi053.stdout:5290 read 0x8194728 thru 0x819803b (0x3914 bytes) 2024-09-13T05:03:06.220 INFO:teuthology.orchestra.run.smithi053.stdout:5292 write 0x77b4583 thru 0x77c3b37 (0xf5b5 bytes) 2024-09-13T05:03:06.222 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.222 INFO:teuthology.orchestra.run.smithi053.stdout:5294 read 0x84a09fc thru 0x84acfb9 (0xc5be bytes) 2024-09-13T05:03:06.222 INFO:teuthology.orchestra.run.smithi053.stdout:5296 punch from 0x48f6555 to 0x48f694a, (0x3f5 bytes) 2024-09-13T05:03:06.222 INFO:teuthology.orchestra.run.smithi053.stdout:5298 write 0x19568c1 thru 0x195f7d5 (0x8f15 bytes) 2024-09-13T05:03:06.224 INFO:teuthology.orchestra.run.smithi053.stdout:5300 punch from 0xbcd7b88 to 0xbcdc291, (0x4709 bytes) 2024-09-13T05:03:06.224 INFO:teuthology.orchestra.run.smithi053.stdout:5301 writesame 0x190334e thru 0x190fefc data_size 0x27(0xcbaf bytes) 2024-09-13T05:03:06.225 INFO:teuthology.orchestra.run.smithi053.stdout:5302 punch from 0xb4aa200 to 0xb4ac465, (0x2265 bytes) 2024-09-13T05:03:06.233 INFO:teuthology.orchestra.run.smithi053.stdout:5303 write 0xd85058b thru 0xd85a4da (0x9f50 bytes) 2024-09-13T05:03:06.250 INFO:teuthology.orchestra.run.smithi053.stdout:5305 trunc from 0xd85a4db to 0x10860c3 2024-09-13T05:03:06.261 INFO:teuthology.orchestra.run.smithi053.stdout:5306 read 0x8241b6 thru 0x82e255 (0xa0a0 bytes) 2024-09-13T05:03:06.262 INFO:teuthology.orchestra.run.smithi053.stdout:5308 punch from 0x840943 to 0x850414, (0xfad1 bytes) 2024-09-13T05:03:06.262 INFO:teuthology.orchestra.run.smithi053.stdout:5309 read 0x68549d thru 0x6935f3 (0xe157 bytes) 2024-09-13T05:03:06.262 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.273 INFO:teuthology.orchestra.run.smithi053.stdout:5311 writesame 0x6836b05 thru 0x683c723 data_size 0x7(0x5c1f bytes) 2024-09-13T05:03:06.285 INFO:teuthology.orchestra.run.smithi053.stdout:5312 writesame 0xc1e4e98 thru 0xc1e891d data_size 0x16(0x3a86 bytes) 2024-09-13T05:03:06.289 INFO:teuthology.orchestra.run.smithi053.stdout:5313 trunc from 0xc1e891e to 0x7eab1a1 2024-09-13T05:03:06.295 INFO:teuthology.orchestra.run.smithi053.stdout:5314 read 0xa5fe65 thru 0xa6cd2e (0xceca bytes) 2024-09-13T05:03:06.295 INFO:teuthology.orchestra.run.smithi053.stdout:5316 read 0x296b2a6 thru 0x2971373 (0x60ce bytes) 2024-09-13T05:03:06.295 INFO:teuthology.orchestra.run.smithi053.stdout:5317 read 0xacc943 thru 0xad8a8b (0xc149 bytes) 2024-09-13T05:03:06.296 INFO:teuthology.orchestra.run.smithi053.stdout:5318 read 0x44d027a thru 0x44d88d5 (0x865c bytes) 2024-09-13T05:03:06.302 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:06.301+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:06.302 INFO:teuthology.orchestra.run.smithi053.stdout:5319 trunc from 0x7eab1a1 to 0x21c2fc2 2024-09-13T05:03:06.309 INFO:teuthology.orchestra.run.smithi053.stdout:5320 read 0x967b4 thru 0x96946 (0x193 bytes) 2024-09-13T05:03:06.309 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.309 INFO:teuthology.orchestra.run.smithi053.stdout:5324 read 0x1bc0de8 thru 0x1bc5683 (0x489c bytes) 2024-09-13T05:03:06.309 INFO:teuthology.orchestra.run.smithi053.stdout:5325 punch from 0xa61dd4 to 0xa6d95d, (0xbb89 bytes) 2024-09-13T05:03:06.309 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.323 INFO:teuthology.orchestra.run.smithi053.stdout:5327 write 0x9e0409c thru 0x9e0a8a3 (0x6808 bytes) 2024-09-13T05:03:06.329 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:06.329 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:06.333 INFO:teuthology.orchestra.run.smithi053.stdout:5328 write 0xa9a83f0 thru 0xa9b53b9 (0xcfca bytes) 2024-09-13T05:03:06.335 INFO:teuthology.orchestra.run.smithi053.stdout:5331 writesame 0x6f6143c thru 0x6f64691 data_size 0x11(0x3256 bytes) 2024-09-13T05:03:06.341 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:06.342 INFO:teuthology.orchestra.run.smithi053.stdout:5333 trunc from 0xa9b53ba to 0x7838ff3 2024-09-13T05:03:06.353 INFO:teuthology.orchestra.run.smithi053.stdout:5335 punch from 0x72e4c42 to 0x72f1059, (0xc417 bytes) 2024-09-13T05:03:06.353 INFO:teuthology.orchestra.run.smithi053.stdout:5336 write 0x544e7c9 thru 0x5455efe (0x7736 bytes) 2024-09-13T05:03:06.356 INFO:teuthology.orchestra.run.smithi053.stdout:5337 read 0x17ccc6f thru 0x17d615e (0x94f0 bytes) 2024-09-13T05:03:06.365 INFO:teuthology.orchestra.run.smithi053.stdout:5339 write 0xa42365c thru 0xa425b8d (0x2532 bytes) 2024-09-13T05:03:06.369 INFO:teuthology.orchestra.run.smithi053.stdout:5340 trunc from 0xa425b8e to 0xac1e8f4 2024-09-13T05:03:06.383 INFO:teuthology.orchestra.run.smithi053.stdout:5341 writesame 0xd8e5f16 thru 0xd8edef6 data_size 0x1(0x7fe1 bytes) 2024-09-13T05:03:06.383 INFO:teuthology.orchestra.run.smithi053.stdout:5342 punch from 0x81e54b2 to 0x81f3a5b, (0xe5a9 bytes) 2024-09-13T05:03:06.383 INFO:teuthology.orchestra.run.smithi053.stdout:5344 writesame 0x75bf1bf thru 0x75cbdf9 data_size 0x4d(0xcc3b bytes) 2024-09-13T05:03:06.385 INFO:teuthology.orchestra.run.smithi053.stdout:5345 write 0x729a259 thru 0x72a24f2 (0x829a bytes) 2024-09-13T05:03:06.387 INFO:teuthology.orchestra.run.smithi053.stdout:5346 read 0xca549c9 thru 0xca5dc8e (0x92c6 bytes) 2024-09-13T05:03:06.387 INFO:teuthology.orchestra.run.smithi053.stdout:5348 write 0x363c325 thru 0x363c908 (0x5e4 bytes) 2024-09-13T05:03:06.390 INFO:teuthology.orchestra.run.smithi053.stdout:5349 read 0x8fb6699 thru 0x8fc20a3 (0xba0b bytes) 2024-09-13T05:03:06.390 INFO:teuthology.orchestra.run.smithi053.stdout:5350 punch from 0x1470f83 to 0x147f7b1, (0xe82e bytes) 2024-09-13T05:03:06.402 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:06.402+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:06.406 INFO:teuthology.orchestra.run.smithi053.stdout:5351 trunc from 0xd8edef7 to 0x19dd1d5 2024-09-13T05:03:06.418 INFO:teuthology.orchestra.run.smithi053.stdout:5352 read 0xd7e2e6 thru 0xd83f01 (0x5c1c bytes) 2024-09-13T05:03:06.419 INFO:teuthology.orchestra.run.smithi053.stdout:5354 read 0x14a6369 thru 0x14ab7b0 (0x5448 bytes) 2024-09-13T05:03:06.419 INFO:teuthology.orchestra.run.smithi053.stdout:5355 read 0x337c63 thru 0x342db8 (0xb156 bytes) 2024-09-13T05:03:06.420 INFO:teuthology.orchestra.run.smithi053.stdout:5356 write 0x8dfef5 thru 0x8ebc33 (0xbd3f bytes) 2024-09-13T05:03:06.437 INFO:teuthology.orchestra.run.smithi053.stdout:5357 writesame 0x8ea9d28 thru 0x8eaf4db data_size 0x6(0x57b4 bytes) 2024-09-13T05:03:06.437 INFO:teuthology.orchestra.run.smithi053.stdout:5358 punch from 0x4a57a3f to 0x4a6400b, (0xc5cc bytes) 2024-09-13T05:03:06.437 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.437 INFO:teuthology.orchestra.run.smithi053.stdout:5360 read 0x5efa190 thru 0x5f09f26 (0xfd97 bytes) 2024-09-13T05:03:06.443 INFO:teuthology.orchestra.run.smithi053.stdout:5361 trunc from 0x8eaf4dc to 0xd7236f2 2024-09-13T05:03:06.449 INFO:teuthology.orchestra.run.smithi053.stdout:5362 punch from 0x349e5a to 0x34e64e, (0x47f4 bytes) 2024-09-13T05:03:06.449 INFO:teuthology.orchestra.run.smithi053.stdout:5363 punch from 0x8467ca5 to 0x847518e, (0xd4e9 bytes) 2024-09-13T05:03:06.449 INFO:teuthology.orchestra.run.smithi053.stdout:5364 read 0x1de7c4b thru 0x1df2c2a (0xafe0 bytes) 2024-09-13T05:03:06.449 INFO:teuthology.orchestra.run.smithi053.stdout:5365 read 0x57a78c7 thru 0x57b56cf (0xde09 bytes) 2024-09-13T05:03:06.449 INFO:teuthology.orchestra.run.smithi053.stdout:5366 write 0x51f590d thru 0x51f7bbb (0x22af bytes) 2024-09-13T05:03:06.452 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.453 INFO:teuthology.orchestra.run.smithi053.stdout:5368 write 0x61e83a thru 0x62c4f0 (0xdcb7 bytes) 2024-09-13T05:03:06.456 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.456 INFO:teuthology.orchestra.run.smithi053.stdout:5371 write 0x1e467fa thru 0x1e4f516 (0x8d1d bytes) 2024-09-13T05:03:06.458 INFO:teuthology.orchestra.run.smithi053.stdout:5372 write 0x6d5b478 thru 0x6d67899 (0xc422 bytes) 2024-09-13T05:03:06.461 INFO:teuthology.orchestra.run.smithi053.stdout:5375 read 0xb5ca07d thru 0xb5cff36 (0x5eba bytes) 2024-09-13T05:03:06.468 INFO:teuthology.orchestra.run.smithi053.stdout:5377 writesame 0xe3b23d9 thru 0xe3bca39 data_size 0x1(0xa661 bytes) 2024-09-13T05:03:06.468 INFO:teuthology.orchestra.run.smithi053.stdout:5378 read 0x99e5551 thru 0x99ebb6c (0x661c bytes) 2024-09-13T05:03:06.468 INFO:teuthology.orchestra.run.smithi053.stdout:5379 writesame 0x63fcdc thru 0x64176b data_size 0x28(0x1a90 bytes) 2024-09-13T05:03:06.469 INFO:teuthology.orchestra.run.smithi053.stdout:5380 write 0x823269b thru 0x8234be0 (0x2546 bytes) 2024-09-13T05:03:06.472 INFO:teuthology.orchestra.run.smithi053.stdout:5381 write 0x6e96258 thru 0x6e9998e (0x3737 bytes) 2024-09-13T05:03:06.474 INFO:teuthology.orchestra.run.smithi053.stdout:5382 write 0x1066528 thru 0x10751f0 (0xecc9 bytes) 2024-09-13T05:03:06.476 INFO:teuthology.orchestra.run.smithi053.stdout:5383 write 0x4a48bc0 thru 0x4a573df (0xe820 bytes) 2024-09-13T05:03:06.477 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.478 INFO:teuthology.orchestra.run.smithi053.stdout:5385 write 0x137d29c thru 0x138d112 (0xfe77 bytes) 2024-09-13T05:03:06.479 INFO:teuthology.orchestra.run.smithi053.stdout:5386 read 0x7796335 thru 0x779f534 (0x9200 bytes) 2024-09-13T05:03:06.479 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:06.480 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:06.480 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.485 INFO:teuthology.orchestra.run.smithi053.stdout:5388 write 0xe474332 thru 0xe4793f0 (0x50bf bytes) 2024-09-13T05:03:06.489 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:06.495 INFO:teuthology.orchestra.run.smithi053.stdout:5389 trunc from 0xe4793f1 to 0x84a863b 2024-09-13T05:03:06.506 INFO:teuthology.orchestra.run.smithi053.stdout:5390 write 0x69ec97c thru 0x69f6256 (0x98db bytes) 2024-09-13T05:03:06.507 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:06.506+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:06.508 INFO:teuthology.orchestra.run.smithi053.stdout:5391 punch from 0x63c5f2f to 0x63c895f, (0x2a30 bytes) 2024-09-13T05:03:06.508 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.508 INFO:teuthology.orchestra.run.smithi053.stdout:5394 write 0x296a890 thru 0x29764e4 (0xbc55 bytes) 2024-09-13T05:03:06.510 INFO:teuthology.orchestra.run.smithi053.stdout:5395 read 0x804c36e thru 0x805154b (0x51de bytes) 2024-09-13T05:03:06.510 INFO:teuthology.orchestra.run.smithi053.stdout:5397 read 0x373872d thru 0x373e238 (0x5b0c bytes) 2024-09-13T05:03:06.511 INFO:teuthology.orchestra.run.smithi053.stdout:5398 write 0x47e7a6a thru 0x47f382d (0xbdc4 bytes) 2024-09-13T05:03:06.512 INFO:teuthology.orchestra.run.smithi053.stdout:5399 write 0x6762b60 thru 0x67653d6 (0x2877 bytes) 2024-09-13T05:03:06.520 INFO:teuthology.orchestra.run.smithi053.stdout:5400 trunc from 0x84a863b to 0x3e825e9 2024-09-13T05:03:06.532 INFO:teuthology.orchestra.run.smithi053.stdout:5401 read 0xac01ca thru 0xac075b (0x592 bytes) 2024-09-13T05:03:06.532 INFO:teuthology.orchestra.run.smithi053.stdout:5402 write 0x1a1f8f0 thru 0x1a2cbda (0xd2eb bytes) 2024-09-13T05:03:06.534 INFO:teuthology.orchestra.run.smithi053.stdout:5404 read 0x14bc3d5 thru 0x14be256 (0x1e82 bytes) 2024-09-13T05:03:06.551 INFO:teuthology.orchestra.run.smithi053.stdout:5407 write 0xd368106 thru 0xd36bb06 (0x3a01 bytes) 2024-09-13T05:03:06.553 INFO:teuthology.orchestra.run.smithi053.stdout:5408 writesame 0x302c3ce thru 0x30326eb data_size 0x3(0x631e bytes) 2024-09-13T05:03:06.566 INFO:teuthology.orchestra.run.smithi053.stdout:5409 trunc from 0xd36bb07 to 0x5a847ce 2024-09-13T05:03:06.582 INFO:teuthology.orchestra.run.smithi053.stdout:5410 trunc from 0x5a847ce to 0xb1104fa 2024-09-13T05:03:06.588 INFO:teuthology.orchestra.run.smithi053.stdout:5411 flatten 2024-09-13T05:03:06.594 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.594 INFO:teuthology.orchestra.run.smithi053.stdout:5413 write 0x8d446b1 thru 0x8d545d6 (0xff26 bytes) 2024-09-13T05:03:06.598 INFO:teuthology.orchestra.run.smithi053.stdout:5414 punch from 0x6ee56ef to 0x6ef19f9, (0xc30a bytes) 2024-09-13T05:03:06.598 INFO:teuthology.orchestra.run.smithi053.stdout:5415 write 0x8234528 thru 0x8241bab (0xd684 bytes) 2024-09-13T05:03:06.601 INFO:teuthology.orchestra.run.smithi053.stdout:5416 punch from 0x4ad7676 to 0x4adbe19, (0x47a3 bytes) 2024-09-13T05:03:06.601 INFO:teuthology.orchestra.run.smithi053.stdout:5417 write 0x1325f01 thru 0x1330241 (0xa341 bytes) 2024-09-13T05:03:06.603 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.603 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.604 INFO:teuthology.orchestra.run.smithi053.stdout:5424 writesame 0x4d781ec thru 0x4d82368 data_size 0x1(0xa17d bytes) 2024-09-13T05:03:06.604 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:06.603+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:06.614 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:06.615 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:06.624 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:06.633 INFO:teuthology.orchestra.run.smithi053.stdout:5425 trunc from 0xb1104fa to 0x3ae3b0f 2024-09-13T05:03:06.653 INFO:teuthology.orchestra.run.smithi053.stdout:5426 trunc from 0x3ae3b0f to 0xbe5fd24 2024-09-13T05:03:06.659 INFO:teuthology.orchestra.run.smithi053.stdout:5427 punch from 0x56ae826 to 0x56be148, (0xf922 bytes) 2024-09-13T05:03:06.669 INFO:teuthology.orchestra.run.smithi053.stdout:5428 trunc from 0xbe5fd24 to 0x303e6d4 2024-09-13T05:03:06.676 INFO:teuthology.orchestra.run.smithi053.stdout:5429 punch from 0x23aa020 to 0x23aa254, (0x234 bytes) 2024-09-13T05:03:06.676 INFO:teuthology.orchestra.run.smithi053.stdout:5431 read 0xcf040f thru 0xcffe2f (0xfa21 bytes) 2024-09-13T05:03:06.688 INFO:teuthology.orchestra.run.smithi053.stdout:5434 write 0x80f0a97 thru 0x80f0e2a (0x394 bytes) 2024-09-13T05:03:06.691 INFO:teuthology.orchestra.run.smithi053.stdout:5435 read 0x5306efc thru 0x530df7f (0x7084 bytes) 2024-09-13T05:03:06.698 INFO:teuthology.orchestra.run.smithi053.stdout:5436 trunc from 0x80f0e2b to 0x22b7fbe 2024-09-13T05:03:06.705 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:06.704+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:06.709 INFO:teuthology.orchestra.run.smithi053.stdout:5439 read 0x119e974 thru 0x11adf19 (0xf5a6 bytes) 2024-09-13T05:03:06.709 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.710 INFO:teuthology.orchestra.run.smithi053.stdout:5444 punch from 0x1ce2bdd to 0x1cea3a5, (0x77c8 bytes) 2024-09-13T05:03:06.710 INFO:teuthology.orchestra.run.smithi053.stdout:5445 read 0x169440a thru 0x169f9ea (0xb5e1 bytes) 2024-09-13T05:03:06.723 INFO:teuthology.orchestra.run.smithi053.stdout:5447 trunc from 0x22b7fbe to 0xdedaad4 2024-09-13T05:03:06.728 INFO:teuthology.orchestra.run.smithi053.stdout:5448 read 0x2cc8e9e thru 0x2cd51ab (0xc30e bytes) 2024-09-13T05:03:06.728 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.738 INFO:teuthology.orchestra.run.smithi053.stdout:5450 trunc from 0xdedaad4 to 0x5a4ee7e 2024-09-13T05:03:06.743 INFO:teuthology.orchestra.run.smithi053.stdout:5451 read 0x1d2ff4e thru 0x1d301a6 (0x259 bytes) 2024-09-13T05:03:06.756 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:06.756 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:06.760 INFO:teuthology.orchestra.run.smithi053.stdout:5452 writesame 0xea6076d thru 0xea687a7 data_size 0x1(0x803b bytes) 2024-09-13T05:03:06.760 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.760 INFO:teuthology.orchestra.run.smithi053.stdout:5454 write 0x86b71e4 thru 0x86c14ac (0xa2c9 bytes) 2024-09-13T05:03:06.763 INFO:teuthology.orchestra.run.smithi053.stdout:5455 writesame 0xa1da56d thru 0xa1dfe66 data_size 0x7(0x58fa bytes) 2024-09-13T05:03:06.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:06.767 INFO:teuthology.orchestra.run.smithi053.stdout:5457 writesame 0x75c4d4e thru 0x75c927d data_size 0x52(0x4530 bytes) 2024-09-13T05:03:06.770 INFO:teuthology.orchestra.run.smithi053.stdout:5458 read 0xab0ad04 thru 0xab1613a (0xb437 bytes) 2024-09-13T05:03:06.775 INFO:teuthology.orchestra.run.smithi053.stdout:5461 writesame 0xeb86a72 thru 0xeb8d64a data_size 0x1(0x6bd9 bytes) 2024-09-13T05:03:06.777 INFO:teuthology.orchestra.run.smithi053.stdout:5465 trunc from 0xeb8d64b to 0xd43889d 2024-09-13T05:03:06.791 INFO:teuthology.orchestra.run.smithi053.stdout:5466 trunc from 0xd43889d to 0x67ff502 2024-09-13T05:03:06.802 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.802 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.802 INFO:teuthology.orchestra.run.smithi053.stdout:5470 read 0x1514065 thru 0x1522294 (0xe230 bytes) 2024-09-13T05:03:06.802 INFO:teuthology.orchestra.run.smithi053.stdout:5472 read 0x475480e thru 0x4758f1f (0x4712 bytes) 2024-09-13T05:03:06.806 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:06.805+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:06.817 INFO:teuthology.orchestra.run.smithi053.stdout:5474 writesame 0xed8b7f0 thru 0xed99239 data_size 0x1(0xda4a bytes) 2024-09-13T05:03:06.817 INFO:teuthology.orchestra.run.smithi053.stdout:5477 write 0x9be0d18 thru 0x9be2ee7 (0x21d0 bytes) 2024-09-13T05:03:06.834 INFO:teuthology.orchestra.run.smithi053.stdout:5478 trunc from 0xed9923a to 0x29f6a68 2024-09-13T05:03:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:5479 read 0x23bece3 thru 0x23c6745 (0x7a63 bytes) 2024-09-13T05:03:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:5481 punch from 0x6f47f2 to 0x6fea37, (0xa245 bytes) 2024-09-13T05:03:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:5483 punch from 0x155da70 to 0x1566089, (0x8619 bytes) 2024-09-13T05:03:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:5484 punch from 0x1126c9b to 0x112bc10, (0x4f75 bytes) 2024-09-13T05:03:06.846 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.860 INFO:teuthology.orchestra.run.smithi053.stdout:5487 write 0xa82f08e thru 0xa83cc58 (0xdbcb bytes) 2024-09-13T05:03:06.863 INFO:teuthology.orchestra.run.smithi053.stdout:5488 write 0x2614396 thru 0x261a52e (0x6199 bytes) 2024-09-13T05:03:06.865 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.872 INFO:teuthology.orchestra.run.smithi053.stdout:5491 writesame 0xc6dcba5 thru 0xc6e8fc2 data_size 0xd(0xc41e bytes) 2024-09-13T05:03:06.873 INFO:teuthology.orchestra.run.smithi053.stdout:5492 write 0xb9b3b7e thru 0xb9b66d0 (0x2b53 bytes) 2024-09-13T05:03:06.876 INFO:teuthology.orchestra.run.smithi053.stdout:5493 write 0x414dc thru 0x4c054 (0xab79 bytes) 2024-09-13T05:03:06.878 INFO:teuthology.orchestra.run.smithi053.stdout:5495 write 0xc4690c5 thru 0xc47120d (0x8149 bytes) 2024-09-13T05:03:06.881 INFO:teuthology.orchestra.run.smithi053.stdout:5496 writesame 0xc23a123 thru 0xc23b68d data_size 0x1(0x156b bytes) 2024-09-13T05:03:06.885 INFO:teuthology.orchestra.run.smithi053.stdout:5497 write 0x4c51be6 thru 0x4c5d917 (0xbd32 bytes) 2024-09-13T05:03:06.887 INFO:teuthology.orchestra.run.smithi053.stdout:5498 write 0xb2d074c thru 0xb2d8ab0 (0x8365 bytes) 2024-09-13T05:03:06.888 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:06.888 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:06.889 INFO:teuthology.orchestra.run.smithi053.stdout:5499 write 0xc6a5abe thru 0xc6ad0a5 (0x75e8 bytes) 2024-09-13T05:03:06.890 INFO:teuthology.orchestra.run.smithi053.stdout:5501 read 0xc8493f thru 0xc8b75c (0x6e1e bytes) 2024-09-13T05:03:06.891 INFO:teuthology.orchestra.run.smithi053.stdout:5502 writesame 0xa1ac764 thru 0xa1b4e93 data_size 0x1c(0x8730 bytes) 2024-09-13T05:03:06.892 INFO:teuthology.orchestra.run.smithi053.stdout:5503 read 0x3c06782 thru 0x3c08b12 (0x2391 bytes) 2024-09-13T05:03:06.893 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.898 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:06.901 INFO:teuthology.orchestra.run.smithi053.stdout:5506 write 0xda8396e thru 0xda8cc67 (0x92fa bytes) 2024-09-13T05:03:06.903 INFO:teuthology.orchestra.run.smithi053.stdout:5507 write 0xeabe1f thru 0xeb2eac (0x708e bytes) 2024-09-13T05:03:06.904 INFO:teuthology.orchestra.run.smithi053.stdout:5508 writesame 0x3684748 thru 0x3686951 data_size 0x1(0x220a bytes) 2024-09-13T05:03:06.907 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:06.906+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:06.908 INFO:teuthology.orchestra.run.smithi053.stdout:5509 write 0x7030343 thru 0x7031c1d (0x18db bytes) 2024-09-13T05:03:06.912 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.912 INFO:teuthology.orchestra.run.smithi053.stdout:5511 write 0xa534386 thru 0xa534e01 (0xa7c bytes) 2024-09-13T05:03:06.913 INFO:teuthology.orchestra.run.smithi053.stdout:5512 punch from 0x28eda7b to 0x28f09d9, (0x2f5e bytes) 2024-09-13T05:03:06.913 INFO:teuthology.orchestra.run.smithi053.stdout:5513 read 0x3c69609 thru 0x3c712b8 (0x7cb0 bytes) 2024-09-13T05:03:06.921 INFO:teuthology.orchestra.run.smithi053.stdout:5514 trunc from 0xda8cc68 to 0x7bb45c1 2024-09-13T05:03:06.939 INFO:teuthology.orchestra.run.smithi053.stdout:5516 writesame 0x7ce5ceb thru 0x7ceb2bb data_size 0x3(0x55d1 bytes) 2024-09-13T05:03:06.939 INFO:teuthology.orchestra.run.smithi053.stdout:5517 punch from 0x2fa192c to 0x2faf169, (0xd83d bytes) 2024-09-13T05:03:06.939 INFO:teuthology.orchestra.run.smithi053.stdout:5518 write 0x2245bf thru 0x230970 (0xc3b2 bytes) 2024-09-13T05:03:06.949 INFO:teuthology.orchestra.run.smithi053.stdout:5519 write 0xbb042bf thru 0xbb0fdcc (0xbb0e bytes) 2024-09-13T05:03:06.965 INFO:teuthology.orchestra.run.smithi053.stdout:5522 trunc from 0xbb0fdcd to 0x6044ea 2024-09-13T05:03:06.976 INFO:teuthology.orchestra.run.smithi053.stdout:5524 read 0x560303 thru 0x5618a7 (0x15a5 bytes) 2024-09-13T05:03:06.976 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.986 INFO:teuthology.orchestra.run.smithi053.stdout:5528 writesame 0x40a32dc thru 0x40a82b1 data_size 0xb(0x4fd6 bytes) 2024-09-13T05:03:06.987 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:06.996 INFO:teuthology.orchestra.run.smithi053.stdout:5531 writesame 0x714e277 thru 0x715322b data_size 0x23(0x4fb5 bytes) 2024-09-13T05:03:06.996 INFO:teuthology.orchestra.run.smithi053.stdout:5532 write 0x231dba thru 0x23923b (0x7482 bytes) 2024-09-13T05:03:06.998 INFO:teuthology.orchestra.run.smithi053.stdout:5534 write 0x524b789 thru 0x5257388 (0xbc00 bytes) 2024-09-13T05:03:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:5535 punch from 0x4f5d4ad to 0x4f69608, (0xc15b bytes) 2024-09-13T05:03:07.001 INFO:teuthology.orchestra.run.smithi053.stdout:5537 read 0x38b13c3 thru 0x38b9243 (0x7e81 bytes) 2024-09-13T05:03:07.002 INFO:teuthology.orchestra.run.smithi053.stdout:5538 punch from 0x3780393 to 0x3785a1a, (0x5687 bytes) 2024-09-13T05:03:07.002 INFO:teuthology.orchestra.run.smithi053.stdout:5541 read 0x3d1f7cd thru 0x3d25828 (0x605c bytes) 2024-09-13T05:03:07.004 INFO:teuthology.orchestra.run.smithi053.stdout:5542 trunc from 0x715322c to 0x8d4fed1 2024-09-13T05:03:07.008 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:07.007+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:07.009 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.009 INFO:teuthology.orchestra.run.smithi053.stdout:5544 trunc from 0x8d4fed1 to 0x8b2bffd 2024-09-13T05:03:07.013 INFO:teuthology.orchestra.run.smithi053.stdout:5546 read 0x3688ce0 thru 0x368edfb (0x611c bytes) 2024-09-13T05:03:07.014 INFO:teuthology.orchestra.run.smithi053.stdout:5547 writesame 0x48a0d13 thru 0x48a2b87 data_size 0x3(0x1e75 bytes) 2024-09-13T05:03:07.016 INFO:teuthology.orchestra.run.smithi053.stdout:5548 read 0x233d933 thru 0x233f904 (0x1fd2 bytes) 2024-09-13T05:03:07.021 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.021 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.026 INFO:teuthology.orchestra.run.smithi053.stdout:5549 write 0xd301ebd thru 0xd3027e6 (0x92a bytes) 2024-09-13T05:03:07.030 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.030 INFO:teuthology.orchestra.run.smithi053.stdout:5551 read 0x8e63dac thru 0x8e67300 (0x3555 bytes) 2024-09-13T05:03:07.031 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:07.042 INFO:teuthology.orchestra.run.smithi053.stdout:5552 trunc from 0xd3027e7 to 0x2acca61 2024-09-13T05:03:07.063 INFO:teuthology.orchestra.run.smithi053.stdout:5553 writesame 0x6666a0a thru 0x6675c92 data_size 0x1(0xf289 bytes) 2024-09-13T05:03:07.063 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.063 INFO:teuthology.orchestra.run.smithi053.stdout:5555 write 0x1dea645 thru 0x1df495b (0xa317 bytes) 2024-09-13T05:03:07.066 INFO:teuthology.orchestra.run.smithi053.stdout:5556 trunc from 0x6675c93 to 0x54da0ef 2024-09-13T05:03:07.071 INFO:teuthology.orchestra.run.smithi053.stdout:5557 punch from 0x51b6e75 to 0x51c1908, (0xaa93 bytes) 2024-09-13T05:03:07.074 INFO:teuthology.orchestra.run.smithi053.stdout:5558 trunc from 0x54da0ef to 0x74f0d7a 2024-09-13T05:03:07.090 INFO:teuthology.orchestra.run.smithi053.stdout:5559 writesame 0xb0f8fdd thru 0xb0ffa9f data_size 0x1(0x6ac3 bytes) 2024-09-13T05:03:07.090 INFO:teuthology.orchestra.run.smithi053.stdout:5560 writesame 0x3048f56 thru 0x304d380 data_size 0x3(0x442b bytes) 2024-09-13T05:03:07.093 INFO:teuthology.orchestra.run.smithi053.stdout:5561 writesame 0x6481dd1 thru 0x6482572 data_size 0x1(0x7a2 bytes) 2024-09-13T05:03:07.097 INFO:teuthology.orchestra.run.smithi053.stdout:5562 read 0x70ae634 thru 0x70ba20a (0xbbd7 bytes) 2024-09-13T05:03:07.097 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.106 INFO:teuthology.orchestra.run.smithi053.stdout:5565 trunc from 0xb0ffaa0 to 0x26b8bfd 2024-09-13T05:03:07.108 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:07.108+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:07.118 INFO:teuthology.orchestra.run.smithi053.stdout:5566 read 0xaf8fa5 thru 0xb0042b (0x7487 bytes) 2024-09-13T05:03:07.131 INFO:teuthology.orchestra.run.smithi053.stdout:5568 write 0x93909c5 thru 0x93998a7 (0x8ee3 bytes) 2024-09-13T05:03:07.143 INFO:teuthology.orchestra.run.smithi053.stdout:5571 writesame 0xc054cb8 thru 0xc060087 data_size 0x30(0xb3d0 bytes) 2024-09-13T05:03:07.143 INFO:teuthology.orchestra.run.smithi053.stdout:5572 write 0x22a3e14 thru 0x22b1c63 (0xde50 bytes) 2024-09-13T05:03:07.145 INFO:teuthology.orchestra.run.smithi053.stdout:5573 read 0x484a2b4 thru 0x484aa2a (0x777 bytes) 2024-09-13T05:03:07.145 INFO:teuthology.orchestra.run.smithi053.stdout:5574 write 0xb390d9 thru 0xb3eaad (0x59d5 bytes) 2024-09-13T05:03:07.160 INFO:teuthology.orchestra.run.smithi053.stdout:5575 trunc from 0xc060088 to 0x10b20f5 2024-09-13T05:03:07.164 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.164 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.172 INFO:teuthology.orchestra.run.smithi053.stdout:5576 punch from 0xa3151a to 0xa35983, (0x4469 bytes) 2024-09-13T05:03:07.172 INFO:teuthology.orchestra.run.smithi053.stdout:5578 punch from 0x495884 to 0x4a162c, (0xbda8 bytes) 2024-09-13T05:03:07.173 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:07.180 INFO:teuthology.orchestra.run.smithi053.stdout:5580 write 0x2fe239d thru 0x2fe6428 (0x408c bytes) 2024-09-13T05:03:07.188 INFO:teuthology.orchestra.run.smithi053.stdout:5581 write 0x3b23b20 thru 0x3b2fa77 (0xbf58 bytes) 2024-09-13T05:03:07.208 INFO:teuthology.orchestra.run.smithi053.stdout:5583 write 0xe351d8d thru 0xe35fb0a (0xdd7e bytes) 2024-09-13T05:03:07.209 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:07.209+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:07.211 INFO:teuthology.orchestra.run.smithi053.stdout:5584 read 0x66a453e thru 0x66b1ef6 (0xd9b9 bytes) 2024-09-13T05:03:07.211 INFO:teuthology.orchestra.run.smithi053.stdout:5585 writesame 0x4c94842 thru 0x4c9ae16 data_size 0x1(0x65d5 bytes) 2024-09-13T05:03:07.225 INFO:teuthology.orchestra.run.smithi053.stdout:5586 write 0x895b3f9 thru 0x89630c8 (0x7cd0 bytes) 2024-09-13T05:03:07.229 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.229 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.229 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.229 INFO:teuthology.orchestra.run.smithi053.stdout:5592 punch from 0x4f3bdfe to 0x4f4812e, (0xc330 bytes) 2024-09-13T05:03:07.229 INFO:teuthology.orchestra.run.smithi053.stdout:5593 writesame 0x86f8f49 thru 0x86fe466 data_size 0x5(0x551e bytes) 2024-09-13T05:03:07.233 INFO:teuthology.orchestra.run.smithi053.stdout:5595 read 0x607fce8 thru 0x608172d (0x1a46 bytes) 2024-09-13T05:03:07.233 INFO:teuthology.orchestra.run.smithi053.stdout:5596 read 0x884dc89 thru 0x884e86e (0xbe6 bytes) 2024-09-13T05:03:07.233 INFO:teuthology.orchestra.run.smithi053.stdout:5598 read 0x2f6e03f thru 0x2f78798 (0xa75a bytes) 2024-09-13T05:03:07.234 INFO:teuthology.orchestra.run.smithi053.stdout:5600 write 0x555ebcc thru 0x5564f44 (0x6379 bytes) 2024-09-13T05:03:07.237 INFO:teuthology.orchestra.run.smithi053.stdout:5601 read 0x6fdf9ec thru 0x6fe02de (0x8f3 bytes) 2024-09-13T05:03:07.237 INFO:teuthology.orchestra.run.smithi053.stdout:5602 read 0x9df3f86 thru 0x9dfe61d (0xa698 bytes) 2024-09-13T05:03:07.237 INFO:teuthology.orchestra.run.smithi053.stdout:5603 write 0xd837670 thru 0xd842fda (0xb96b bytes) 2024-09-13T05:03:07.240 INFO:teuthology.orchestra.run.smithi053.stdout:5604 read 0xa38a722 thru 0xa394733 (0xa012 bytes) 2024-09-13T05:03:07.240 INFO:teuthology.orchestra.run.smithi053.stdout:5605 read 0xb025843 thru 0xb0335e8 (0xdda6 bytes) 2024-09-13T05:03:07.241 INFO:teuthology.orchestra.run.smithi053.stdout:5606 write 0x37da63a thru 0x37dbb31 (0x14f8 bytes) 2024-09-13T05:03:07.242 INFO:teuthology.orchestra.run.smithi053.stdout:5609 write 0xaa9afa4 thru 0xaaa783a (0xc897 bytes) 2024-09-13T05:03:07.245 INFO:teuthology.orchestra.run.smithi053.stdout:5610 writesame 0xc42d8a6 thru 0xc43a99d data_size 0x12(0xd0f8 bytes) 2024-09-13T05:03:07.248 INFO:teuthology.orchestra.run.smithi053.stdout:5611 read 0x586447f thru 0x58743b4 (0xff36 bytes) 2024-09-13T05:03:07.248 INFO:teuthology.orchestra.run.smithi053.stdout:5613 write 0xd9fbfde thru 0xd9fe24d (0x2270 bytes) 2024-09-13T05:03:07.249 INFO:teuthology.orchestra.run.smithi053.stdout:5614 read 0xdb8eae thru 0xdc2699 (0x97ec bytes) 2024-09-13T05:03:07.250 INFO:teuthology.orchestra.run.smithi053.stdout:5615 write 0x7605f37 thru 0x7611a6a (0xbb34 bytes) 2024-09-13T05:03:07.253 INFO:teuthology.orchestra.run.smithi053.stdout:5616 writesame 0xe1bbd38 thru 0xe1bc77b data_size 0x12(0xa44 bytes) 2024-09-13T05:03:07.254 INFO:teuthology.orchestra.run.smithi053.stdout:5617 write 0x136b63d thru 0x13796b3 (0xe077 bytes) 2024-09-13T05:03:07.256 INFO:teuthology.orchestra.run.smithi053.stdout:5618 punch from 0x647a016 to 0x64857c8, (0xb7b2 bytes) 2024-09-13T05:03:07.257 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.257 INFO:teuthology.orchestra.run.smithi053.stdout:5623 punch from 0xa00aa04 to 0xa00e7d2, (0x3dce bytes) 2024-09-13T05:03:07.257 INFO:teuthology.orchestra.run.smithi053.stdout:5624 read 0x15d603f thru 0x15e0b46 (0xab08 bytes) 2024-09-13T05:03:07.257 INFO:teuthology.orchestra.run.smithi053.stdout:5625 read 0xa198937 thru 0xa19bc8b (0x3355 bytes) 2024-09-13T05:03:07.257 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.258 INFO:teuthology.orchestra.run.smithi053.stdout:5628 write 0x45cc13c thru 0x45d9510 (0xd3d5 bytes) 2024-09-13T05:03:07.259 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.259 INFO:teuthology.orchestra.run.smithi053.stdout:5632 write 0x2de0137 thru 0x2deb782 (0xb64c bytes) 2024-09-13T05:03:07.261 INFO:teuthology.orchestra.run.smithi053.stdout:5633 writesame 0xc58ce88 thru 0xc593f16 data_size 0x55(0x708f bytes) 2024-09-13T05:03:07.262 INFO:teuthology.orchestra.run.smithi053.stdout:5634 punch from 0x5471a20 to 0x547796c, (0x5f4c bytes) 2024-09-13T05:03:07.263 INFO:teuthology.orchestra.run.smithi053.stdout:5636 trunc from 0xe35fb0b to 0xd8b17f2 2024-09-13T05:03:07.276 INFO:teuthology.orchestra.run.smithi053.stdout:5637 write 0xe3e2044 thru 0xe3e624a (0x4207 bytes) 2024-09-13T05:03:07.277 INFO:teuthology.orchestra.run.smithi053.stdout:5638 write 0xc432077 thru 0xc440fc9 (0xef53 bytes) 2024-09-13T05:03:07.279 INFO:teuthology.orchestra.run.smithi053.stdout:5639 punch from 0x84d0ca3 to 0x84de555, (0xd8b2 bytes) 2024-09-13T05:03:07.280 INFO:teuthology.orchestra.run.smithi053.stdout:5641 punch from 0x9177aca to 0x917fff9, (0x852f bytes) 2024-09-13T05:03:07.280 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.280 INFO:teuthology.orchestra.run.smithi053.stdout:5645 write 0xb0f91fd thru 0xb107a17 (0xe81b bytes) 2024-09-13T05:03:07.281 INFO:teuthology.orchestra.run.smithi053.stdout:5646 writesame 0xcac9037 thru 0xcacb542 data_size 0x2(0x250c bytes) 2024-09-13T05:03:07.284 INFO:teuthology.orchestra.run.smithi053.stdout:5647 write 0x67646f5 thru 0x677030c (0xbc18 bytes) 2024-09-13T05:03:07.285 INFO:teuthology.orchestra.run.smithi053.stdout:5650 read 0x856d300 thru 0x856def0 (0xbf1 bytes) 2024-09-13T05:03:07.293 INFO:teuthology.orchestra.run.smithi053.stdout:5652 trunc from 0xe3e624b to 0x812e6d0 2024-09-13T05:03:07.302 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.302 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.304 INFO:teuthology.orchestra.run.smithi053.stdout:5654 read 0x517433f thru 0x517958a (0x524c bytes) 2024-09-13T05:03:07.310 INFO:teuthology.orchestra.run.smithi053.stdout:5655 trunc from 0x812e6d0 to 0xcd43968 2024-09-13T05:03:07.310 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:07.310+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:07.314 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:07.314 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:07.314 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:07.314 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T05:03:07.314 INFO:tasks.thrashosds.thrasher:Setting osd 6 primary_affinity to 0.000000 2024-09-13T05:03:07.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 0 2024-09-13T05:03:07.323 INFO:teuthology.orchestra.run.smithi053.stdout:5657 write 0xdf61607 thru 0xdf6e8bb (0xd2b5 bytes) 2024-09-13T05:03:07.326 INFO:teuthology.orchestra.run.smithi053.stdout:5658 read 0x7c685e7 thru 0x7c70ac8 (0x84e2 bytes) 2024-09-13T05:03:07.327 INFO:teuthology.orchestra.run.smithi053.stdout:5659 read 0x272ac81 thru 0x2739e60 (0xf1e0 bytes) 2024-09-13T05:03:07.327 INFO:teuthology.orchestra.run.smithi053.stdout:5662 read 0x1ffaa51 thru 0x20079f1 (0xcfa1 bytes) 2024-09-13T05:03:07.327 INFO:teuthology.orchestra.run.smithi053.stdout:5664 write 0x3a238a1 thru 0x3a26386 (0x2ae6 bytes) 2024-09-13T05:03:07.338 INFO:teuthology.orchestra.run.smithi053.stdout:5665 trunc from 0xdf6e8bc to 0x5b4d994 2024-09-13T05:03:07.351 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.362 INFO:teuthology.orchestra.run.smithi053.stdout:5667 write 0xb31fff5 thru 0xb3283b2 (0x83be bytes) 2024-09-13T05:03:07.365 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.365 INFO:teuthology.orchestra.run.smithi053.stdout:5669 punch from 0x9ac5c63 to 0x9ac855e, (0x28fb bytes) 2024-09-13T05:03:07.368 INFO:teuthology.orchestra.run.smithi053.stdout:5670 trunc from 0xb3283b3 to 0xaaa274e 2024-09-13T05:03:07.372 INFO:teuthology.orchestra.run.smithi053.stdout:5671 writesame 0x3f4d691 thru 0x3f5c29f data_size 0x59(0xec0f bytes) 2024-09-13T05:03:07.379 INFO:teuthology.orchestra.run.smithi053.stdout:5672 trunc from 0xaaa274e to 0x585eb82 2024-09-13T05:03:07.391 INFO:teuthology.orchestra.run.smithi053.stdout:5673 read 0x1a1769b thru 0x1a17bf3 (0x559 bytes) 2024-09-13T05:03:07.407 INFO:teuthology.orchestra.run.smithi053.stdout:5674 writesame 0xe9a67b7 thru 0xe9b22ae data_size 0x7c(0xbaf8 bytes) 2024-09-13T05:03:07.407 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.408 INFO:teuthology.orchestra.run.smithi053.stdout:5677 punch from 0x553ea13 to 0x55499eb, (0xafd8 bytes) 2024-09-13T05:03:07.408 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.408 INFO:teuthology.orchestra.run.smithi053.stdout:5680 write 0xd9ff62c thru 0xda0658b (0x6f60 bytes) 2024-09-13T05:03:07.411 INFO:teuthology.orchestra.run.smithi053.stdout:5681 read 0x967e862 thru 0x9687410 (0x8baf bytes) 2024-09-13T05:03:07.411 INFO:teuthology.orchestra.run.smithi053.stdout:5682 read 0xd78b4f0 thru 0xd78f8e9 (0x43fa bytes) 2024-09-13T05:03:07.411 INFO:teuthology.orchestra.run.smithi053.stdout:5683 write 0x5f50197 thru 0x5f52bb9 (0x2a23 bytes) 2024-09-13T05:03:07.412 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:07.411+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:07.412 INFO:teuthology.orchestra.run.smithi053.stdout:5684 writesame 0x22ed132 thru 0x22fb738 data_size 0x1b(0xe607 bytes) 2024-09-13T05:03:07.415 INFO:teuthology.orchestra.run.smithi053.stdout:5685 write 0xbfb8cdb thru 0xbfbfe0f (0x7135 bytes) 2024-09-13T05:03:07.418 INFO:teuthology.orchestra.run.smithi053.stdout:5687 write 0x90d3beb thru 0x90d4515 (0x92b bytes) 2024-09-13T05:03:07.419 INFO:teuthology.orchestra.run.smithi053.stdout:5688 read 0x1bc5d04 thru 0x1bcb787 (0x5a84 bytes) 2024-09-13T05:03:07.420 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:07.420 INFO:teuthology.orchestra.run.smithi053.stdout:5691 writesame 0x862b216 thru 0x862c7bc data_size 0x1(0x15a7 bytes) 2024-09-13T05:03:07.423 INFO:teuthology.orchestra.run.smithi053.stdout:5693 write 0x6198996 thru 0x61a7ea0 (0xf50b bytes) 2024-09-13T05:03:07.425 INFO:teuthology.orchestra.run.smithi053.stdout:5694 write 0x77d31f thru 0x78bafe (0xe7e0 bytes) 2024-09-13T05:03:07.426 INFO:teuthology.orchestra.run.smithi053.stdout:5695 clone 37 order 19 su 65536 sc 14 2024-09-13T05:03:07.444 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.444 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:07.539 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:07.538+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:07.602 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.602 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:07.652 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:07.651+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:07.713 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:07.713+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:07.756 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.757 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:07.814 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:07.814+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:07.871 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.872 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.880 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:07.916 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:07.915+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:07.927 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.6 primary-affinity to 0 (802) 2024-09-13T05:03:07.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:03:07.989 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:07.989 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:07.998 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:08.042 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:08.041+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:08.101 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:08.101 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:08.109 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:08.151 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:08.150+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:08.205 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:08.218 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:08.217+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:08.252 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:08.252 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:08.260 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:08.329 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:08.329 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:08.345 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:08.344+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:08.345 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:08.345 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:08.361 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:08.361 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:08.377 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:08.377 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:08.394 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:08.394 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:08.403 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:08.403 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:08.410 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:08.410 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:08.412 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:08.418 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:08.418+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:08.426 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:08.427 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:08.443 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:08.443 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:08.520 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:08.519+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:08.557 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:08.557 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:08.557 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:08.557 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:08.557 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:08.565 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:08.650 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:08.649+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:08.710 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:08.710 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091740151, 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.711 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:08.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.715 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091721572000001, 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:08.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.717 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.720 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091696777999999, 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.721 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:08.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091677422, 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:08.730 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091658735999999, 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:08.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091633667, 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.735 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.739 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091611292, 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.740 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:08.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:08.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.744 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091590474, 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:08.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:08.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:08.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091565170000001, 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.750 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.754 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091546445000001, 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091528384, 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:08.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091496153, 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:08.764 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:08.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:08.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091476993000001, 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:08.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091459891, 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.773 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.774 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:08.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091433594, 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.778 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:08.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091407747, 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.783 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:08.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.787 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091390895, 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.788 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.792 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091374535, 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:08.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091358411, 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.091337181, 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:08.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:08.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:08.806 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:08.806 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:08.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:08.808 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:08.807+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:08.908 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:08.908+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:08.932 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:03:08.954 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:08.954 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:08.954 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:08.963 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:09.009 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:09.009+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:09.066 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:09.066 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:09.066 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:09.066 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:09.066 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:09.075 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:09.110 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:09.110+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:09.146 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-13T05:03:09.161 INFO:teuthology.orchestra.run.smithi053.stdout:5696 trunc from 0xe9b22af to 0x5c96af7 2024-09-13T05:03:09.186 INFO:teuthology.orchestra.run.smithi053.stdout:5697 punch from 0x2d4901a to 0x2d54495, (0xb47b bytes) 2024-09-13T05:03:09.187 INFO:teuthology.orchestra.run.smithi053.stdout:5698 read 0x2e0cc4e thru 0x2e17121 (0xa4d4 bytes) 2024-09-13T05:03:09.187 INFO:teuthology.orchestra.run.smithi053.stdout:5700 read 0x21b6149 thru 0x21c1a27 (0xb8df bytes) 2024-09-13T05:03:09.188 INFO:teuthology.orchestra.run.smithi053.stdout:5701 read 0x3146dcb thru 0x314ee1f (0x8055 bytes) 2024-09-13T05:03:09.188 INFO:teuthology.orchestra.run.smithi053.stdout:5702 read 0x24a78a7 thru 0x24ac77b (0x4ed5 bytes) 2024-09-13T05:03:09.188 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:09.188 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:09.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:09.188 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70864 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.607241348000001, 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038688958000000002, 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70864 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.189 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579270+0000", 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.491e-06 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579283+0000", 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2653999999999999e-05 2024-09-13T05:03:09.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579289+0000", 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.614870+0000", 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035581387999999999 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615507+0000", 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00063753200000000001 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617957+0000", 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024498380000000002 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70866 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2596864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.193 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.607050988999999, 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036734241000000001, 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70866 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:09.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579458+0000", 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579462+0000", 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579471+0000", 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9639999999999992e-06 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579477+0000", 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0000000000000002e-06 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615818+0000", 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036340823000000001 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616117+0000", 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029901500000000002 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616193+0000", 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5829999999999997e-05 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70869 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.607026751999999, 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036867536999999999, 2024-09-13T05:03:09.199 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70869 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579482+0000", 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579485+0000", 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3329999999999999e-06 2024-09-13T05:03:09.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579491+0000", 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1580000000000002e-06 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579496+0000", 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8679999999999998e-06 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616223+0000", 2024-09-13T05:03:09.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036727187000000001 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616290+0000", 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7160000000000001e-05 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616350+0000", 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0248999999999999e-05 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70877 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.606470429, 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036468836999999997, 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70877 2024-09-13T05:03:09.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.205 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580043+0000", 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3989999999999997e-06 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580052+0000", 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6319999999999997e-06 2024-09-13T05:03:09.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580065+0000", 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3288000000000001e-05 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616372+0000", 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036307147999999997 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616447+0000", 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5296999999999994e-05 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616508+0000", 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0751999999999997e-05 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70879 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1130496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.209 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.605723587, 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035893590000000003, 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70879 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580785+0000", 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580792+0000", 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9029999999999998e-06 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580804+0000", 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1545999999999999e-05 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580811+0000", 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2609999999999996e-06 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616530+0000", 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035719133 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616604+0000", 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3712000000000002e-05 2024-09-13T05:03:09.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616680+0000", 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5957000000000005e-05 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70882 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.605692121000001, 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036032559999999998, 2024-09-13T05:03:09.214 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70882 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580817+0000", 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580820+0000", 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5720000000000001e-06 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580826+0000", 2024-09-13T05:03:09.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4130000000000003e-06 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580831+0000", 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.865e-06 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616703+0000", 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035871694000000003 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616782+0000", 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9369999999999994e-05 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616850+0000", 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8118000000000001e-05 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70891 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1089536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.219 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.605673028, 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036169077000000001, 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70891 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:09.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580836+0000", 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580839+0000", 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3760000000000002e-06 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580853+0000", 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4735999999999999e-05 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580859+0000", 2024-09-13T05:03:09.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3329999999999996e-06 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616872+0000", 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036013495999999999 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616945+0000", 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2646999999999997e-05 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617006+0000", 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0853000000000002e-05 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70893 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.605639202999999, 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036286807999999997, 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70893 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580873+0000", 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7930000000000002e-06 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580880+0000", 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9970000000000001e-06 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580885+0000", 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5350000000000004e-06 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036141860999999997 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617091+0000", 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3730000000000001e-05 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6412e-05 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.229 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70895 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1187840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.605528451, 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036346900000000001, 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70895 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.230 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580985+0000", 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4009999999999999e-06 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581000+0000", 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5024999999999999e-05 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581006+0000", 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3080000000000004e-06 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617178+0000", 2024-09-13T05:03:09.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036171571999999999 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617328+0000", 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1537000000000002e-05 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.234 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71006 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.600926137, 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035821812000000001, 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.235 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71006 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:09.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585583+0000", 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585586+0000", 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585593+0000", 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9210000000000003e-06 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585598+0000", 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0159999999999999e-06 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621279+0000", 2024-09-13T05:03:09.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035681069000000003 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621346+0000", 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7080000000000001e-05 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621405+0000", 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9401999999999998e-05 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.239 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71008 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.600904876, 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035951295000000001, 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71008 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.240 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585604+0000", 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585607+0000", 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.266e-06 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585613+0000", 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585622+0000", 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7590000000000003e-06 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621431+0000", 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035809331 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621495+0000", 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4611999999999997e-05 2024-09-13T05:03:09.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621556+0000", 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0591999999999999e-05 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71009 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:09.244 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.600883475, 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036076001000000003, 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71009 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585634+0000", 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.055e-06 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585640+0000", 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4189999999999999e-06 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585646+0000", 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621580+0000", 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035934440999999998 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621644+0000", 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3927e-05 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621702+0000", 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7855000000000001e-05 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71011 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.600854326, 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036193110000000001, 2024-09-13T05:03:09.249 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71011 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585658+0000", 2024-09-13T05:03:09.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7329999999999999e-06 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585666+0000", 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7280000000000001e-06 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585671+0000", 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-06 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621726+0000", 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036054614999999998 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621790+0000", 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3868000000000006e-05 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621848+0000", 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8913000000000003e-05 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.253 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71071 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.59994521, 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035743500999999997, 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.254 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71071 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586566+0000", 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3209999999999999e-06 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586572+0000", 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586578+0000", 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622184+0000", 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035606024999999999 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622248+0000", 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4691999999999997e-05 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622308+0000", 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9663000000000001e-05 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71096 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.598970422000001, 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035817952, 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71096 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:09.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587541+0000", 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587548+0000", 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4679999999999997e-06 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.261 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587561+0000", 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2713e-05 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623235+0000", 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035674375000000001 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623300+0000", 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4505999999999995e-05 2024-09-13T05:03:09.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623357+0000", 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7661999999999998e-05 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71097 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.598941355999999, 2024-09-13T05:03:09.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035944619999999997, 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71097 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587575+0000", 2024-09-13T05:03:09.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587582+0000", 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1160000000000001e-06 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587587+0000", 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0490000000000003e-06 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623383+0000", 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035795897 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623448+0000", 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4604999999999996e-05 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623513+0000", 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5168999999999999e-05 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.267 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71115 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.598113746999999, 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036023959000000001, 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.268 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71115 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588395+0000", 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588400+0000", 2024-09-13T05:03:09.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588410+0000", 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0076e-05 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588417+0000", 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6549999999999998e-06 2024-09-13T05:03:09.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623694+0000", 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035277159000000002 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623760+0000", 2024-09-13T05:03:09.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5968000000000003e-05 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624420+0000", 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066017099999999998 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71117 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.597865354, 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035938036, 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71117 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588643+0000", 2024-09-13T05:03:09.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588649+0000", 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5210000000000002e-06 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588660+0000", 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1459e-05 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588668+0000", 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5669999999999997e-06 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624447+0000", 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035778841999999998 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624516+0000", 2024-09-13T05:03:09.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8774000000000003e-05 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624582+0000", 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6818999999999998e-05 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71119 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:09.278 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.597546187000001, 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035764241000000002, 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71119 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588962+0000", 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588968+0000", 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5770000000000001e-06 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588980+0000", 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1718e-05 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588987+0000", 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624607+0000", 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035619545000000002 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624670+0000", 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3664999999999994e-05 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624728+0000", 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7433999999999999e-05 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.282 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71120 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1220608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 14.597365452, 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035736393999999998, 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.283 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71120 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589143+0000", 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589149+0000", 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1329999999999998e-06 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589167+0000", 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8236999999999998e-05 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589174+0000", 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4889999999999998e-06 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624752+0000", 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035577723999999998 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624813+0000", 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1138999999999999e-05 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624881+0000", 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7489000000000004e-05 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.287 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:09.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:09.291 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:09.288+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:09.293 INFO:teuthology.orchestra.run.smithi053.stdout:5704 writesame 0x97b23e5 thru 0x97b7806 data_size 0x16(0x5422 bytes) 2024-09-13T05:03:09.293 INFO:teuthology.orchestra.run.smithi053.stdout:5706 writesame 0xc98b736 thru 0xc996212 data_size 0x1f(0xaadd bytes) 2024-09-13T05:03:09.293 INFO:teuthology.orchestra.run.smithi053.stdout:5707 write 0xa3ed834 thru 0xa3f43d2 (0x6b9f bytes) 2024-09-13T05:03:09.293 INFO:teuthology.orchestra.run.smithi053.stdout:5708 write 0x57486e8 thru 0x5753e2a (0xb743 bytes) 2024-09-13T05:03:09.293 INFO:teuthology.orchestra.run.smithi053.stdout:5709 write 0xcd82eeb thru 0xcd8df96 (0xb0ac bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5710 read 0x48473b7 thru 0x484c89f (0x54e9 bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5711 write 0x5527712 thru 0x5529332 (0x1c21 bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5712 write 0x6efdd22 thru 0x6f0adae (0xd08d bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5713 read 0xa68d4bf thru 0xa69c4da (0xf01c bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5714 write 0x712c91d thru 0x71341ed (0x78d1 bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5715 read 0xa1a29e2 thru 0xa1a6100 (0x371f bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5716 writesame 0x52662f1 thru 0x526b583 data_size 0x1(0x5293 bytes) 2024-09-13T05:03:09.294 INFO:teuthology.orchestra.run.smithi053.stdout:5717 read 0x44aae04 thru 0x44b6ec5 (0xc0c2 bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5718 read 0x8360f8 thru 0x83e415 (0x831e bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5719 trunc from 0xcd8df97 to 0x7ce0437 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5720 read 0x7462597 thru 0x7462abc (0x526 bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5722 punch from 0x51a0da1 to 0x51a5348, (0x45a7 bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5723 read 0x5ddba6d thru 0x5dea3cd (0xe961 bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5724 writesame 0x9a9a3c9 thru 0x9aa46bc data_size 0x2(0xa2f4 bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5726 punch from 0x71e681d to 0x71ea9ae, (0x4191 bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5727 read 0x99aeead thru 0x99b86fb (0x984f bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5728 write 0xb47681c thru 0xb482fb5 (0xc79a bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5730 writesame 0x7c86a59 thru 0x7c95d56 data_size 0x13(0xf2fe bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5731 read 0x12ee828 thru 0x12fb8b2 (0xd08b bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5732 write 0x3620b99 thru 0x3622a27 (0x1e8f bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5733 punch from 0x1b1bc70 to 0x1b1c90e, (0xc9e bytes) 2024-09-13T05:03:09.295 INFO:teuthology.orchestra.run.smithi053.stdout:5734 punch from 0x8f0c10f to 0x8f163e2, (0xa2d3 bytes) 2024-09-13T05:03:09.296 INFO:teuthology.orchestra.run.smithi053.stdout:5737 writesame 0x3456499 thru 0x345fbb9 data_size 0x1(0x9721 bytes) 2024-09-13T05:03:09.314 INFO:teuthology.orchestra.run.smithi053.stdout:5738 writesame 0xd7f267b thru 0xd7fbc58 data_size 0x1(0x95de bytes) 2024-09-13T05:03:09.317 INFO:teuthology.orchestra.run.smithi053.stdout:5739 trunc from 0xd7fbc59 to 0xb08b516 2024-09-13T05:03:09.323 INFO:teuthology.orchestra.run.smithi053.stdout:5740 read 0x9523c04 thru 0x9529715 (0x5b12 bytes) 2024-09-13T05:03:09.328 INFO:teuthology.orchestra.run.smithi053.stdout:5742 trunc from 0xb08b516 to 0x6c5ea8d 2024-09-13T05:03:09.339 INFO:teuthology.orchestra.run.smithi053.stdout:5743 punch from 0x1ca405b to 0x1cae21f, (0xa1c4 bytes) 2024-09-13T05:03:09.339 INFO:teuthology.orchestra.run.smithi053.stdout:5744 punch from 0x50bef0a to 0x50c5385, (0x647b bytes) 2024-09-13T05:03:09.339 INFO:teuthology.orchestra.run.smithi053.stdout:5745 writesame 0x214eea2 thru 0x215e68d data_size 0x6(0xf7ec bytes) 2024-09-13T05:03:09.349 INFO:teuthology.orchestra.run.smithi053.stdout:5749 read 0x436a46a thru 0x43724ca (0x8061 bytes) 2024-09-13T05:03:09.358 INFO:teuthology.orchestra.run.smithi053.stdout:5751 writesame 0xad8ddb7 thru 0xad930f9 data_size 0x31(0x5343 bytes) 2024-09-13T05:03:09.358 INFO:teuthology.orchestra.run.smithi053.stdout:5752 read 0x98e472f thru 0x98e5830 (0x1102 bytes) 2024-09-13T05:03:09.358 INFO:teuthology.orchestra.run.smithi053.stdout:5753 read 0x3288d7 thru 0x334360 (0xba8a bytes) 2024-09-13T05:03:09.368 INFO:teuthology.orchestra.run.smithi053.stdout:5755 writesame 0xdf8fce2 thru 0xdf9ac71 data_size 0x6a(0xaf90 bytes) 2024-09-13T05:03:09.368 INFO:teuthology.orchestra.run.smithi053.stdout:5756 read 0x4e985ec thru 0x4e9fac3 (0x74d8 bytes) 2024-09-13T05:03:09.368 INFO:teuthology.orchestra.run.smithi053.stdout:5758 punch from 0x48d1e59 to 0x48d3d98, (0x1f3f bytes) 2024-09-13T05:03:09.368 INFO:teuthology.orchestra.run.smithi053.stdout:5759 writesame 0x1748864 thru 0x1753346 data_size 0x29(0xaae3 bytes) 2024-09-13T05:03:09.373 INFO:teuthology.orchestra.run.smithi053.stdout:5760 write 0x7df7a35 thru 0x7dfea3c (0x7008 bytes) 2024-09-13T05:03:09.376 INFO:teuthology.orchestra.run.smithi053.stdout:5761 write 0x179c30f thru 0x17a0bdd (0x48cf bytes) 2024-09-13T05:03:09.381 INFO:teuthology.orchestra.run.smithi053.stdout:5762 read 0x6e887aa thru 0x6e920ae (0x9905 bytes) 2024-09-13T05:03:09.381 INFO:teuthology.orchestra.run.smithi053.stdout:5763 read 0x22bb176 thru 0x22bc5ea (0x1475 bytes) 2024-09-13T05:03:09.382 INFO:teuthology.orchestra.run.smithi053.stdout:5765 trunc from 0xdf9ac72 to 0xd354784 2024-09-13T05:03:09.387 INFO:teuthology.orchestra.run.smithi053.stdout:5766 write 0xd27d73 thru 0xd36eba (0xf148 bytes) 2024-09-13T05:03:09.389 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:09.389+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:09.392 INFO:teuthology.orchestra.run.smithi053.stdout:5768 write 0x4613ad5 thru 0x461c41e (0x894a bytes) 2024-09-13T05:03:09.396 INFO:teuthology.orchestra.run.smithi053.stdout:5769 write 0x5eb71bf thru 0x5ebed1f (0x7b61 bytes) 2024-09-13T05:03:09.398 INFO:teuthology.orchestra.run.smithi053.stdout:5770 punch from 0x7b2259c to 0x7b2bb01, (0x9565 bytes) 2024-09-13T05:03:09.398 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.398 INFO:teuthology.orchestra.run.smithi053.stdout:5772 read 0x34c70d0 thru 0x34cb303 (0x4234 bytes) 2024-09-13T05:03:09.411 INFO:teuthology.orchestra.run.smithi053.stdout:5774 trunc from 0xd354784 to 0x1b7a6c0 2024-09-13T05:03:09.441 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:09.441 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:09.441 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:09.441 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:09.446 INFO:teuthology.orchestra.run.smithi053.stdout:5775 write 0x7009026 thru 0x70165c7 (0xd5a2 bytes) 2024-09-13T05:03:09.450 INFO:teuthology.orchestra.run.smithi053.stdout:5776 read 0x574700f thru 0x574a601 (0x35f3 bytes) 2024-09-13T05:03:09.451 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:09.461 INFO:teuthology.orchestra.run.smithi053.stdout:5777 write 0xc310a22 thru 0xc312f02 (0x24e1 bytes) 2024-09-13T05:03:09.464 INFO:teuthology.orchestra.run.smithi053.stdout:5778 punch from 0xb651469 to 0xb6533be, (0x1f55 bytes) 2024-09-13T05:03:09.473 INFO:teuthology.orchestra.run.smithi053.stdout:5780 trunc from 0xc312f03 to 0x4fd53af 2024-09-13T05:03:09.483 INFO:teuthology.orchestra.run.smithi053.stdout:5782 read 0x3083878 thru 0x308ca65 (0x91ee bytes) 2024-09-13T05:03:09.483 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.483 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.491 INFO:teuthology.orchestra.run.smithi053.stdout:5786 trunc from 0x4fd53af to 0xbc63dc6 2024-09-13T05:03:09.494 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:09.494+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:09.496 INFO:teuthology.orchestra.run.smithi053.stdout:5787 read 0x3453779 thru 0x3459358 (0x5be0 bytes) 2024-09-13T05:03:09.496 INFO:teuthology.orchestra.run.smithi053.stdout:5788 read 0x604593f thru 0x604d17a (0x783c bytes) 2024-09-13T05:03:09.504 INFO:teuthology.orchestra.run.smithi053.stdout:5790 writesame 0xdf6bc2d thru 0xdf752c5 data_size 0x47(0x9699 bytes) 2024-09-13T05:03:09.504 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.504 INFO:teuthology.orchestra.run.smithi053.stdout:5792 writesame 0x63205db thru 0x63217d9 data_size 0x1(0x11ff bytes) 2024-09-13T05:03:09.508 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.508 INFO:teuthology.orchestra.run.smithi053.stdout:5794 punch from 0x7edebba to 0x7ee72c6, (0x870c bytes) 2024-09-13T05:03:09.508 INFO:teuthology.orchestra.run.smithi053.stdout:5795 read 0xaea9394 thru 0xaead30a (0x3f77 bytes) 2024-09-13T05:03:09.509 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.509 INFO:teuthology.orchestra.run.smithi053.stdout:5798 write 0x70a1cb6 thru 0x70b093c (0xec87 bytes) 2024-09-13T05:03:09.512 INFO:teuthology.orchestra.run.smithi053.stdout:5799 writesame 0x7761600 thru 0x776e2ca data_size 0x1(0xcccb bytes) 2024-09-13T05:03:09.535 INFO:teuthology.orchestra.run.smithi053.stdout:5806 punch from 0x243ea75 to 0x2443388, (0x4913 bytes) 2024-09-13T05:03:09.537 INFO:teuthology.orchestra.run.smithi053.stdout:5807 trunc from 0xdf752c6 to 0xb4ed376 2024-09-13T05:03:09.544 INFO:teuthology.orchestra.run.smithi053.stdout:5808 trunc from 0xb4ed376 to 0xaeedbd5 2024-09-13T05:03:09.549 INFO:teuthology.orchestra.run.smithi053.stdout:5809 write 0x601eee6 thru 0x602dc25 (0xed40 bytes) 2024-09-13T05:03:09.558 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:09.558 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:09.558 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:09.558 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:09.558 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:09.563 INFO:teuthology.orchestra.run.smithi053.stdout:5810 writesame 0xeaa5800 thru 0xeaaf01b data_size 0x84(0x981c bytes) 2024-09-13T05:03:09.563 INFO:teuthology.orchestra.run.smithi053.stdout:5811 punch from 0x43b989c to 0x43c0339, (0x6a9d bytes) 2024-09-13T05:03:09.563 INFO:teuthology.orchestra.run.smithi053.stdout:5813 writesame 0x29237e6 thru 0x29311da data_size 0x7(0xd9f5 bytes) 2024-09-13T05:03:09.567 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:09.569 INFO:teuthology.orchestra.run.smithi053.stdout:5814 punch from 0x64dcc7e to 0x64e2031, (0x53b3 bytes) 2024-09-13T05:03:09.569 INFO:teuthology.orchestra.run.smithi053.stdout:5815 write 0x3ceb202 thru 0x3cec9b4 (0x17b3 bytes) 2024-09-13T05:03:09.572 INFO:teuthology.orchestra.run.smithi053.stdout:5817 punch from 0x8a80581 to 0x8a84046, (0x3ac5 bytes) 2024-09-13T05:03:09.572 INFO:teuthology.orchestra.run.smithi053.stdout:5818 writesame 0x6883151 thru 0x68887d3 data_size 0x1(0x5683 bytes) 2024-09-13T05:03:09.584 INFO:teuthology.orchestra.run.smithi053.stdout:5819 read 0x57a7b6f thru 0x57b59ee (0xde80 bytes) 2024-09-13T05:03:09.585 INFO:teuthology.orchestra.run.smithi053.stdout:5821 read 0x2dd5b4c thru 0x2ddda98 (0x7f4d bytes) 2024-09-13T05:03:09.585 INFO:teuthology.orchestra.run.smithi053.stdout:5823 read 0xcac3880 thru 0xcace24a (0xa9cb bytes) 2024-09-13T05:03:09.585 INFO:teuthology.orchestra.run.smithi053.stdout:5824 read 0x4d6be42 thru 0x4d792ad (0xd46c bytes) 2024-09-13T05:03:09.585 INFO:teuthology.orchestra.run.smithi053.stdout:5826 read 0x64d3420 thru 0x64e0ee0 (0xdac1 bytes) 2024-09-13T05:03:09.585 INFO:teuthology.orchestra.run.smithi053.stdout:5827 write 0x3a8c4f4 thru 0x3a912a8 (0x4db5 bytes) 2024-09-13T05:03:09.588 INFO:teuthology.orchestra.run.smithi053.stdout:5828 punch from 0x477e9a5 to 0x478b10f, (0xc76a bytes) 2024-09-13T05:03:09.593 INFO:teuthology.orchestra.run.smithi053.stdout:5829 trunc from 0xeaaf01c to 0xae7bb30 2024-09-13T05:03:09.597 INFO:teuthology.orchestra.run.smithi053.stdout:5830 write 0x91cbaf thru 0x91e87f (0x1cd1 bytes) 2024-09-13T05:03:09.609 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:09.608+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:09.610 INFO:teuthology.orchestra.run.smithi053.stdout:5831 trunc from 0xae7bb30 to 0x2d2c9ed 2024-09-13T05:03:09.635 INFO:teuthology.orchestra.run.smithi053.stdout:5832 writesame 0xb25db8c thru 0xb262605 data_size 0x1(0x4a7a bytes) 2024-09-13T05:03:09.645 INFO:teuthology.orchestra.run.smithi053.stdout:5833 writesame 0xeb87373 thru 0xeb90679 data_size 0x7(0x9307 bytes) 2024-09-13T05:03:09.645 INFO:teuthology.orchestra.run.smithi053.stdout:5834 read 0x30195ea thru 0x3023db9 (0xa7d0 bytes) 2024-09-13T05:03:09.645 INFO:teuthology.orchestra.run.smithi053.stdout:5835 read 0xd9068ab thru 0xd90bba1 (0x52f7 bytes) 2024-09-13T05:03:09.645 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.645 INFO:teuthology.orchestra.run.smithi053.stdout:5837 read 0x669932e thru 0x66a39a2 (0xa675 bytes) 2024-09-13T05:03:09.646 INFO:teuthology.orchestra.run.smithi053.stdout:5838 read 0xb45be6f thru 0xb461f93 (0x6125 bytes) 2024-09-13T05:03:09.646 INFO:teuthology.orchestra.run.smithi053.stdout:5839 write 0x62f7ed5 thru 0x62fbc60 (0x3d8c bytes) 2024-09-13T05:03:09.649 INFO:teuthology.orchestra.run.smithi053.stdout:5840 trunc from 0xeb9067a to 0xdb2cd88 2024-09-13T05:03:09.654 INFO:teuthology.orchestra.run.smithi053.stdout:5841 write 0xd400ee6 thru 0xd40c71e (0xb839 bytes) 2024-09-13T05:03:09.658 INFO:teuthology.orchestra.run.smithi053.stdout:5842 read 0x9a92322 thru 0x9aa1c80 (0xf95f bytes) 2024-09-13T05:03:09.658 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.671 INFO:teuthology.orchestra.run.smithi053.stdout:5845 trunc from 0xdb2cd88 to 0x264edc5 2024-09-13T05:03:09.694 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:09.693+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:09.695 INFO:teuthology.orchestra.run.smithi053.stdout:5846 writesame 0x7b945e6 thru 0x7ba2b70 data_size 0x1(0xe58b bytes) 2024-09-13T05:03:09.695 INFO:teuthology.orchestra.run.smithi053.stdout:5848 read 0x20f3729 thru 0x20ffb9a (0xc472 bytes) 2024-09-13T05:03:09.695 INFO:teuthology.orchestra.run.smithi053.stdout:5849 read 0x1c82313 thru 0x1c8f11e (0xce0c bytes) 2024-09-13T05:03:09.696 INFO:teuthology.orchestra.run.smithi053.stdout:5850 trunc from 0x7ba2b71 to 0x8667b41 2024-09-13T05:03:09.702 INFO:teuthology.orchestra.run.smithi053.stdout:5851 punch from 0xed5e1e to 0xed6b79, (0xd5b bytes) 2024-09-13T05:03:09.702 INFO:teuthology.orchestra.run.smithi053.stdout:5852 punch from 0x2d9b8ce to 0x2dab421, (0xfb53 bytes) 2024-09-13T05:03:09.702 INFO:teuthology.orchestra.run.smithi053.stdout:5854 write 0x398ddf1 thru 0x3996dcb (0x8fdb bytes) 2024-09-13T05:03:09.706 INFO:teuthology.orchestra.run.smithi053.stdout:5855 read 0x2e3c16c thru 0x2e435f7 (0x748c bytes) 2024-09-13T05:03:09.707 INFO:teuthology.orchestra.run.smithi053.stdout:5856 read 0x462fb89 thru 0x4630bae (0x1026 bytes) 2024-09-13T05:03:09.707 INFO:teuthology.orchestra.run.smithi053.stdout:5859 write 0x4cc19f0 thru 0x4ccbd84 (0xa395 bytes) 2024-09-13T05:03:09.710 INFO:teuthology.orchestra.run.smithi053.stdout:5860 write 0x38bc366 thru 0x38cba44 (0xf6df bytes) 2024-09-13T05:03:09.713 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.713 INFO:teuthology.orchestra.run.smithi053.stdout:5862 read 0x37ca340 thru 0x37cae6b (0xb2c bytes) 2024-09-13T05:03:09.717 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.48729994, 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:09.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:09.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.486371831, 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:09.723 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:09.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:09.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.728 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.485273421999999, 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.729 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:09.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:09.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.479213461000001, 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:09.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:09.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.739 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.479204465999999, 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.743 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.479196094999999, 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:09.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:09.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.748 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.479186073999998, 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.749 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:09.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:09.753 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.479170351, 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:09.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.477268182000003, 2024-09-13T05:03:09.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.763 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.477260223000002, 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:09.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:09.768 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.768 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.477252534999998, 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:09.769 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:09.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:09.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.477241843000002, 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:09.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:09.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.477233601999998, 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:09.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.477209940999998, 2024-09-13T05:03:09.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:09.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.787 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.475567482000002, 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:09.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.475559249, 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:09.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.475336519999999, 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.798 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:09.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:09.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.475323971000002, 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.803 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:09.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.473284745000001, 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:09.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:09.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 45.473274965999998, 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:09.814 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:09.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:09.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:09.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:09.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:09.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.820 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.821 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:09.821 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:09.821 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:09.826 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:09.824+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:09.827 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:09.828 INFO:teuthology.orchestra.run.smithi053.stdout:5864 trunc from 0x8667b41 to 0xc69d6c3 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:5866 read 0x12c72e7 thru 0x12d2702 (0xb41c bytes) 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:5867 writesame 0xb7e90e3 thru 0xb7f124a data_size 0x65(0x8168 bytes) 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:5868 write 0x36bf07e thru 0x36c8344 (0x92c7 bytes) 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:5869 write 0x5ca7c51 thru 0x5cab142 (0x34f2 bytes) 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:5870 read 0x156538d thru 0x1569c4f (0x48c3 bytes) 2024-09-13T05:03:09.829 INFO:teuthology.orchestra.run.smithi053.stdout:5871 writesame 0x8af156c thru 0x8b01455 data_size 0x2(0xfeea bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5873 read 0x673bbe1 thru 0x6746953 (0xad73 bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5874 writesame 0x39e86a1 thru 0x39f2c05 data_size 0x1(0xa565 bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5875 writesame 0xcec9ce9 thru 0xced6d9f data_size 0x11(0xd0b7 bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5876 trunc from 0xced6da0 to 0x1c54fac 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5877 read 0x23175a thru 0x23c6ea (0xaf91 bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5878 read 0x1b72d83 thru 0x1b7a8cc (0x7b4a bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5879 trunc from 0x1c54fac to 0xa1c7fdd 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5880 write 0x81df45b thru 0x81e299f (0x3545 bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5881 punch from 0x1dc804a to 0x1dd33c6, (0xb37c bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5882 punch from 0x862536f to 0x862e50c, (0x919d bytes) 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5885 trunc from 0xa1c7fdd to 0xea3db0d 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.830 INFO:teuthology.orchestra.run.smithi053.stdout:5887 read 0x15750d7 thru 0x15794a7 (0x43d1 bytes) 2024-09-13T05:03:09.831 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.831 INFO:teuthology.orchestra.run.smithi053.stdout:5889 read 0x8c9b984 thru 0x8ca2e41 (0x74be bytes) 2024-09-13T05:03:09.831 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.831 INFO:teuthology.orchestra.run.smithi053.stdout:5891 writesame 0x4360b24 thru 0x436ae1c data_size 0x1(0xa2f9 bytes) 2024-09-13T05:03:09.846 INFO:teuthology.orchestra.run.smithi053.stdout:5892 write 0xaab01dc thru 0xaabf7bf (0xf5e4 bytes) 2024-09-13T05:03:09.850 INFO:teuthology.orchestra.run.smithi053.stdout:5893 write 0xa457b15 thru 0xa45d6b2 (0x5b9e bytes) 2024-09-13T05:03:09.853 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.853 INFO:teuthology.orchestra.run.smithi053.stdout:5896 write 0x92de7d3 thru 0x92ded5d (0x58b bytes) 2024-09-13T05:03:09.856 INFO:teuthology.orchestra.run.smithi053.stdout:5897 read 0x528c789 thru 0x529b0ac (0xe924 bytes) 2024-09-13T05:03:09.856 INFO:teuthology.orchestra.run.smithi053.stdout:5900 read 0xcade70a thru 0xcae4d9e (0x6695 bytes) 2024-09-13T05:03:09.856 INFO:teuthology.orchestra.run.smithi053.stdout:5901 write 0x580ee1c thru 0x581489f (0x5a84 bytes) 2024-09-13T05:03:09.859 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.859 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.860 INFO:teuthology.orchestra.run.smithi053.stdout:5904 punch from 0xdf5adb6 to 0xdf68711, (0xd95b bytes) 2024-09-13T05:03:09.860 INFO:teuthology.orchestra.run.smithi053.stdout:5905 read 0x24eed58 thru 0x24f43b0 (0x5659 bytes) 2024-09-13T05:03:09.860 INFO:teuthology.orchestra.run.smithi053.stdout:5906 write 0x1316fa thru 0x13ce81 (0xb788 bytes) 2024-09-13T05:03:09.863 INFO:teuthology.orchestra.run.smithi053.stdout:5907 punch from 0xc85fff2 to 0xc86d480, (0xd48e bytes) 2024-09-13T05:03:09.869 INFO:teuthology.orchestra.run.smithi053.stdout:5909 trunc from 0xea3db0d to 0x94a7ecd 2024-09-13T05:03:09.882 INFO:teuthology.orchestra.run.smithi053.stdout:5910 trunc from 0x94a7ecd to 0xbc5af06 2024-09-13T05:03:09.888 INFO:teuthology.orchestra.run.smithi053.stdout:5912 write 0x2db5d3a thru 0x2dc477b (0xea42 bytes) 2024-09-13T05:03:09.892 INFO:teuthology.orchestra.run.smithi053.stdout:5913 writesame 0xfa7b26 thru 0xfaf89b data_size 0x6a(0x7d76 bytes) 2024-09-13T05:03:09.896 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.896 INFO:teuthology.orchestra.run.smithi053.stdout:5915 read 0x170ed2 thru 0x17fe0f (0xef3e bytes) 2024-09-13T05:03:09.896 INFO:teuthology.orchestra.run.smithi053.stdout:5916 read 0x43b8529 thru 0x43c6a29 (0xe501 bytes) 2024-09-13T05:03:09.896 INFO:teuthology.orchestra.run.smithi053.stdout:5918 read 0x6e3cffb thru 0x6e4b13e (0xe144 bytes) 2024-09-13T05:03:09.896 INFO:teuthology.orchestra.run.smithi053.stdout:5919 writesame 0x4fa028 thru 0x50841e data_size 0x31(0xe3f7 bytes) 2024-09-13T05:03:09.901 INFO:teuthology.orchestra.run.smithi053.stdout:5920 write 0xb63228a thru 0xb637ee5 (0x5c5c bytes) 2024-09-13T05:03:09.913 INFO:teuthology.orchestra.run.smithi053.stdout:5921 writesame 0xeaac188 thru 0xeab347f data_size 0x34(0x72f8 bytes) 2024-09-13T05:03:09.914 INFO:teuthology.orchestra.run.smithi053.stdout:5922 read 0x4ff6f97 thru 0x4ff7bcb (0xc35 bytes) 2024-09-13T05:03:09.914 INFO:teuthology.orchestra.run.smithi053.stdout:5923 write 0xcb241c1 thru 0xcb30795 (0xc5d5 bytes) 2024-09-13T05:03:09.926 INFO:teuthology.orchestra.run.smithi053.stdout:5926 trunc from 0xeab3480 to 0x8439785 2024-09-13T05:03:09.926 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:09.925+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:09.947 INFO:teuthology.orchestra.run.smithi053.stdout:5927 write 0xa38caa9 thru 0xa38cec1 (0x419 bytes) 2024-09-13T05:03:09.949 INFO:teuthology.orchestra.run.smithi053.stdout:5929 read 0x74f3b2d thru 0x74fc2b9 (0x878d bytes) 2024-09-13T05:03:09.957 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:09.957 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:09.957 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:09.957 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:09.957 INFO:teuthology.orchestra.run.smithi053.stdout:5930 writesame 0xbcff5da thru 0xbd09710 data_size 0x1(0xa137 bytes) 2024-09-13T05:03:09.957 INFO:teuthology.orchestra.run.smithi053.stdout:5931 write 0x9afee21 thru 0x9b0d915 (0xeaf5 bytes) 2024-09-13T05:03:09.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:09.968 INFO:teuthology.orchestra.run.smithi053.stdout:5934 trunc from 0xbd09711 to 0x66206df 2024-09-13T05:03:09.977 INFO:teuthology.orchestra.run.smithi053.stdout:5935 punch from 0x45b8b4e to 0x45bd230, (0x46e2 bytes) 2024-09-13T05:03:09.978 INFO:teuthology.orchestra.run.smithi053.stdout:5936 trunc from 0x66206df to 0x6c7f9cb 2024-09-13T05:03:09.982 INFO:teuthology.orchestra.run.smithi053.stdout:5937 write 0x3d77f31 thru 0x3d7eb44 (0x6c14 bytes) 2024-09-13T05:03:09.986 INFO:teuthology.orchestra.run.smithi053.stdout:5939 read 0x4171d00 thru 0x4172e49 (0x114a bytes) 2024-09-13T05:03:09.987 INFO:teuthology.orchestra.run.smithi053.stdout:5940 read 0x455f809 thru 0x456e326 (0xeb1e bytes) 2024-09-13T05:03:09.987 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:09.987 INFO:teuthology.orchestra.run.smithi053.stdout:5943 read 0x2b8efec thru 0x2b9deef (0xef04 bytes) 2024-09-13T05:03:09.987 INFO:teuthology.orchestra.run.smithi053.stdout:5945 punch from 0x158cb22 to 0x159bbbe, (0xf09c bytes) 2024-09-13T05:03:09.987 INFO:teuthology.orchestra.run.smithi053.stdout:5947 read 0x1c831be thru 0x1c86388 (0x31cb bytes) 2024-09-13T05:03:09.987 INFO:teuthology.orchestra.run.smithi053.stdout:5948 read 0x1871dad thru 0x187ed6a (0xcfbe bytes) 2024-09-13T05:03:09.992 INFO:teuthology.orchestra.run.smithi053.stdout:5951 write 0x78ce957 thru 0x78d0e29 (0x24d3 bytes) 2024-09-13T05:03:09.997 INFO:teuthology.orchestra.run.smithi053.stdout:5952 read 0x4dfc39f thru 0x4e0ac80 (0xe8e2 bytes) 2024-09-13T05:03:09.997 INFO:teuthology.orchestra.run.smithi053.stdout:5953 writesame 0xa6c7e0 thru 0xa71adf data_size 0x10(0x5300 bytes) 2024-09-13T05:03:10.001 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:10.002 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:10.002 INFO:teuthology.orchestra.run.smithi053.stdout:5957 punch from 0x1ef1610 to 0x1eff3d7, (0xddc7 bytes) 2024-09-13T05:03:10.002 INFO:teuthology.orchestra.run.smithi053.stdout:5958 read 0x2dcd3a2 thru 0x2dd6097 (0x8cf6 bytes) 2024-09-13T05:03:10.002 INFO:teuthology.orchestra.run.smithi053.stdout:5959 write 0x785f2e1 thru 0x786ce47 (0xdb67 bytes) 2024-09-13T05:03:10.006 INFO:teuthology.orchestra.run.smithi053.stdout:5961 read 0x23a8404 thru 0x23a9d6d (0x196a bytes) 2024-09-13T05:03:10.016 INFO:teuthology.orchestra.run.smithi053.stdout:5963 write 0xc0a66f1 thru 0xc0ae24e (0x7b5e bytes) 2024-09-13T05:03:10.029 INFO:teuthology.orchestra.run.smithi053.stdout:5965 write 0xd7bb343 thru 0xd7c8299 (0xcf57 bytes) 2024-09-13T05:03:10.029 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:10.028+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:10.032 INFO:teuthology.orchestra.run.smithi053.stdout:5968 trunc from 0xd7c829a to 0xbd28ee9 2024-09-13T05:03:10.043 INFO:teuthology.orchestra.run.smithi053.stdout:5969 write 0x17310c9 thru 0x173a29e (0x91d6 bytes) 2024-09-13T05:03:10.045 INFO:teuthology.orchestra.run.smithi053.stdout:5971 write 0x2d50866 thru 0x2d5bf41 (0xb6dc bytes) 2024-09-13T05:03:10.048 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:10.048 INFO:teuthology.orchestra.run.smithi053.stdout:5973 read 0x21e3ce5 thru 0x21e3cff (0x1b bytes) 2024-09-13T05:03:10.048 INFO:teuthology.orchestra.run.smithi053.stdout:5975 write 0x608a67d thru 0x608f52b (0x4eaf bytes) 2024-09-13T05:03:10.051 INFO:teuthology.orchestra.run.smithi053.stdout:5976 write 0x6282dcd thru 0x628ab52 (0x7d86 bytes) 2024-09-13T05:03:10.055 INFO:teuthology.orchestra.run.smithi053.stdout:5979 trunc from 0xbd28ee9 to 0xcab5fd1 2024-09-13T05:03:10.060 INFO:teuthology.orchestra.run.smithi053.stdout:5980 write 0x1d03475 thru 0x1d0553c (0x20c8 bytes) 2024-09-13T05:03:10.063 INFO:teuthology.orchestra.run.smithi053.stdout:5981 read 0x4c1ebd0 thru 0x4c2138b (0x27bc bytes) 2024-09-13T05:03:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:compare and write disabled 2024-09-13T05:03:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:5985 punch from 0x229ff2c to 0x22a1b51, (0x1c25 bytes) 2024-09-13T05:03:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:5987 punch from 0x2b9a6f3 to 0x2ba4f3c, (0xa849 bytes) 2024-09-13T05:03:10.064 INFO:teuthology.orchestra.run.smithi053.stdout:5989 writesame 0x6ba612c thru 0x6bb4d93 data_size 0xaa(0xec68 bytes) 2024-09-13T05:03:10.067 INFO:teuthology.orchestra.run.smithi053.stdout:5990 writesame 0x63c445c thru 0x63c98d5 data_size 0x2(0x547a bytes) 2024-09-13T05:03:10.074 INFO:teuthology.orchestra.run.smithi053.stdout:5992 writesame 0x4cbbc32 thru 0x4cc493e data_size 0x1(0x8d0d bytes) 2024-09-13T05:03:10.086 INFO:teuthology.orchestra.run.smithi053.stdout:5993 writesame 0x107b363 thru 0x107e57b data_size 0x2d(0x3219 bytes) 2024-09-13T05:03:10.091 INFO:teuthology.orchestra.run.smithi053.stdout:5994 write 0x68d0dce thru 0x68d8b44 (0x7d77 bytes) 2024-09-13T05:03:10.094 INFO:teuthology.orchestra.run.smithi053.stdout:5995 write 0x23ca491 thru 0x23d6e02 (0xc972 bytes) 2024-09-13T05:03:10.097 INFO:teuthology.orchestra.run.smithi053.stdout:5996 clone 38 order 23 su 1048576 sc 15 2024-09-13T05:03:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:10.100 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:10.100 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:10.109 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:10.130 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:10.130+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:10.233 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:10.233+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:10.258 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:10.258 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:10.258 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:10.258 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:10.258 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.258 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.601045466999999, 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.259 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.260 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:03:10.261 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.262 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.60102947, 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.263 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:10.264 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:03:10.265 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.266 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:10.267 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.601012674000003, 2024-09-13T05:03:10.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:03:10.268 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.268 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.268 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:10.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.273 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600995701000002, 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.277 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:10.278 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600979307000003, 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:10.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600962271, 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:03:10.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600928688000003, 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:03:10.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600909559000002, 2024-09-13T05:03:10.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:03:10.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.60083169, 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:03:10.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600770945000001, 2024-09-13T05:03:10.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.301 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:03:10.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600710730999999, 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.305 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:03:10.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600693403999998, 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.309 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:10.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600664232, 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:03:10.313 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:03:10.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:03:10.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:10.317 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600644547999998, 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:03:10.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:10.321 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600627115000002, 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:10.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.323 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600609472999999, 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.326 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:03:10.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600575296999999, 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:10.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:03:10.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:03:10.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:10.335 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600549946999998, 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:03:10.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600532274000003, 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.340 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.343 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 38.600508069999997, 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:03:10.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:10.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.347 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.348 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:10.348 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:10.348 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:10.349 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:10.350 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:10.349+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:10.451 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:10.450+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:10.454 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:10.498 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:10.498 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:10.498 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:10.498 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:10.507 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:10.580 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:10.580 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:10.581 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:10.579+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:10.598 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:10.598 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:10.630 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:10.630 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:10.646 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:10.647 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:10.652 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:10.651+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:10.678 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:10.679 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:10.685 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:10.686 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:10.698 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:10.704 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:10.705 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:10.723 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:10.723 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:10.740 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:10.740 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:10.752 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:10.752+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:10.813 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:10.813 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:10.821 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:10.853 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:10.852+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:10.928 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:10.929 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:10.939 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:10.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:03:10.954 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:10.954+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:10.994 INFO:teuthology.orchestra.run.smithi053.stdout:truncating image image_client.0-clone37 from 0xcab5fd1 (overlap 0x1b7a6c0) to 0x57ee26 2024-09-13T05:03:11.055 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:11.055+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:11.080 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.080 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.091 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:11.106 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-13T05:03:11.156 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:11.155+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:11.240 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.240 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.250 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:11.257 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:11.256+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:11.357 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:11.357+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:11.360 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.361 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.364 INFO:teuthology.orchestra.run.smithi053.stdout:5997 trunc from 0xcab5fd1 to 0xbf759f9 2024-09-13T05:03:11.371 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:11.387 INFO:teuthology.orchestra.run.smithi053.stdout:5998 read 0x467fd84 thru 0x4684ae3 (0x4d60 bytes) 2024-09-13T05:03:11.387 INFO:teuthology.orchestra.run.smithi053.stdout:5999 write 0x821942a thru 0x821d90b (0x44e2 bytes) 2024-09-13T05:03:11.391 INFO:teuthology.orchestra.run.smithi053.stdout:6000 write 0x313221e thru 0x3135f07 (0x3cea bytes) 2024-09-13T05:03:11.425 INFO:teuthology.orchestra.run.smithi053.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-13T05:03:11.458 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:11.458+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:11.488 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.488 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:11.559 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:11.559+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:11.645 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.645 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:11.697 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:11.696+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:11.756 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.756 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.762 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:11.761+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:11.765 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:11.865 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:11.865+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:11.907 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:11.907 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:11.916 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:11.941 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:03:11.966 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:11.965+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:12.033 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.033 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.042 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:12.069 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:12.069+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:12.170 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:12.169+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:12.185 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.185 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.193 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:12.272 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:12.270+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:12.295 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.296 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.304 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:12.371 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:12.371+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:12.408 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.408 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.418 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:12.472 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:12.471+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:12.521 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.521 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.529 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:12.573 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:12.573+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:12.631 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.631 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.640 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:12.674 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:12.674+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:12.743 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.743 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:12.752 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:12.797 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:12.796+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:12.854 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:12.855 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:12.863 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:12.877 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:12.876+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:12.888 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:12.888 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:12.905 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:12.905 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:12.921 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:12.921 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:12.938 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:12.938 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:12.943 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:12.943 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:12.943 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-13T05:03:12.943 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:03:12.943 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:03:12.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-13T05:03:12.961 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:12.961 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:12.978 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:12.977+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:12.981 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:12.982 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:13.000 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:13.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:13.018 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:13.018 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:13.020 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:13.021 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:13.027 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:13.071 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:13.072 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T05:03:13.072 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:13.111 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:13.110+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:13.170 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:13.170 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:13.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:13.170 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.940076607000002, 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.171 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:13.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.939148498000002, 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:13.175 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.179 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.938050089000001, 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:13.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.931990128000002, 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:13.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:13.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:13.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:13.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:13.191 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.931981133000001, 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:13.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.195 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.931972762000001, 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.200 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.931962741, 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:13.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.204 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.931947018000002, 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.206 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:13.210 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.930044848999998, 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:13.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.214 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.930036889999997, 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.215 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:13.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:13.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.930029202, 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.219 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:13.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:13.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.930018509999996, 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:13.223 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:13.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.227 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.930010269, 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:13.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:13.231 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.929986608, 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.232 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:13.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.928344148999997, 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:13.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.928335916000002, 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.241 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.928113187000001, 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.246 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:13.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.928100637999997, 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.250 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:13.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.254 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.926061412000003, 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:13.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:13.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:13.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.260 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 48.926051633, 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.261 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:13.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:13.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:13.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:13.266 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:13.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:13.269 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:13.267+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:13.369 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:13.369+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:13.381 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:13.381 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:13.381 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:13.390 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:13.470 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:13.469+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:13.497 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:13.497 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:13.497 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:13.497 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:13.498 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:13.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:13.571 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:13.570+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957147872, 2024-09-13T05:03:13.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:03:13.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957131875000002, 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.619 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957115078999998, 2024-09-13T05:03:13.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957098105999997, 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:03:13.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957081711999997, 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.632 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:13.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957064676000002, 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:03:13.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957031092999998, 2024-09-13T05:03:13.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:03:13.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.957011964000003, 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:03:13.644 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:03:13.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:03:13.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956934095000001, 2024-09-13T05:03:13.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:03:13.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:13.652 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956873350000002, 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:03:13.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:13.656 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956813136000001, 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:03:13.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:03:13.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956795808999999, 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:03:13.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956766637000001, 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.664 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956746953, 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.668 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:03:13.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956729520000003, 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.672 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.674 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956711878, 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.676 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:13.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:03:13.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:03:13.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956677702, 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.680 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956652351999999, 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:03:13.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956634678999997, 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.689 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:03:13.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:03:13.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:13.693 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.956610474999998, 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:13.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:13.697 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:13.699 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:13.699 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:13.698+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:13.800 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:13.799+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:13.847 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:13.848 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:13.848 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:13.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:13.901 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:13.900+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:13.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:03:13.968 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:13.968 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:13.969 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:13.969 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:13.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:14.001 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:14.001+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:14.102 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:14.102+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775381066999998, 2024-09-13T05:03:14.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:03:14.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775358023999999, 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775340692, 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:03:14.152 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775324148000003, 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:03:14.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:03:14.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775307023000003, 2024-09-13T05:03:14.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:03:14.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775290794999997, 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:03:14.165 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775273984000002, 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.166 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:03:14.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:03:14.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775257365999998, 2024-09-13T05:03:14.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:03:14.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.174 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775240787999998, 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.175 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:14.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775204434000003, 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.179 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:14.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.183 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775185489999998, 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:03:14.187 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775168856000001, 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.188 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775142854999999, 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.192 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.194 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.195 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.196 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775110916000003, 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.197 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.198 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.199 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:03:14.200 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775092129000001, 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.201 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.202 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:03:14.203 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.204 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:14.205 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775076009000003, 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.206 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:14.207 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.208 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.209 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775053948, 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.210 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:14.211 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.212 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.213 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.775036983, 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.214 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.215 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.216 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:03:14.217 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.218 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.463002584000002, 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.219 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.220 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.221 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.222 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.462983475999998, 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:03:14.223 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.224 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.225 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.226 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.227 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:14.229 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:14.229 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:14.228+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:14.329 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:14.329+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:14.376 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:14.376 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:14.377 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:14.377 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:14.385 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:14.469 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:14.469+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:14.531 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:14.530+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:14.532 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:14.532 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:14.532 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:14.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:14.632 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:14.632+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:03:14.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.709567720999999, 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:03:14.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:03:14.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:03:14.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:03:14.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 46.884183665999998, 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:03:14.695 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:03:14.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:03:14.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:03:14.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:03:14.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.743114632000001, 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.702 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:03:14.703 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.704 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:03:14.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:03:14.705 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.705 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.705 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.705 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:03:14.705 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.706 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40645 3.4 e37/19 3:2244ece7:::rbd_data.117c4a06f7bc.00000000000001c5:head v 37'221, mlcod=37'219)", 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.511593982000001, 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001776096, 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.707 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:03:14.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.708 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.708 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.708 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.708 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176038+0000", 2024-09-13T05:03:14.708 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176037+0000", 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176044+0000", 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7299999999999999e-06 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176059+0000", 2024-09-13T05:03:14.709 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4953999999999999e-05 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176066+0000", 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7279999999999995e-06 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176109+0000", 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2951000000000003e-05 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.176128+0000", 2024-09-13T05:03:14.710 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8484000000000001e-05 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177768+0000", 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016398700000000001 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177814+0000", 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6159000000000003e-05 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.711 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.510618309000002, 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.712 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:03:14.713 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.714 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:03:14.715 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.510297432000002, 2024-09-13T05:03:14.716 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:03:14.717 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.718 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:03:14.719 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:03:14.720 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.721 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 42.508209475000001, 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:14.722 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.723 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.724 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.725 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:03:14.725 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:14.725 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.726 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:03:14.727 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.413900300000002, 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.728 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.729 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:03:14.730 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.731 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.732 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.108761839000003, 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:14.733 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.734 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.735 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.736 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52876 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'388, mlcod=44'385)", 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.349448437, 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922389, 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.737 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.738 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338182+0000", 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338190+0000", 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.345e-06 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338204+0000", 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4399999999999999e-05 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.739 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338212+0000", 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3880000000000002e-06 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338273+0000", 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1030999999999999e-05 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.740 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338293+0000", 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472e-05 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340073+0000", 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017803719999999999 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340106+0000", 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2186e-05 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.741 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52897 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'391, mlcod=44'388)", 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.344046106, 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001898648, 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343585+0000", 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343592+0000", 2024-09-13T05:03:14.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9110000000000004e-06 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343602+0000", 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343609+0000", 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9099999999999999e-06 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343673+0000", 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4048999999999996e-05 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343692+0000", 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8946000000000001e-05 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345449+0000", 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017568970000000001 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345484+0000", 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5033999999999997e-05 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.329647764999997, 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.746 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.747 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:03:14.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.328360762999999, 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:03:14.754 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52997 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'399, mlcod=44'395)", 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 34.327164570000001, 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001838309, 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.755 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360466+0000", 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360473+0000", 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6150000000000002e-06 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360484+0000", 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1549000000000001e-05 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360495+0000", 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-13T05:03:14.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361021+0000", 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000526035 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361040+0000", 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8482e-05 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362269+0000", 2024-09-13T05:03:14.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012290960000000001 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362305+0000", 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6733000000000002e-05 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.759 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83276 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'726, mlcod=59'724)", 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.712090888000001, 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039477289999999996, 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975540+0000", 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976050+0000", 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050985799999999997 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976069+0000", 2024-09-13T05:03:14.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8393000000000002e-05 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976098+0000", 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9386e-05 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976193+0000", 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4728999999999994e-05 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976217+0000", 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979453+0000", 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032362860000000001 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.764 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979488+0000", 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5629000000000001e-05 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83277 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'728, mlcod=59'724)", 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.978527+0000", 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.709104977999999, 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781955, 2024-09-13T05:03:14.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978527+0000", 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978527+0000", 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978525+0000", 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978535+0000", 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1779999999999992e-06 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978548+0000", 2024-09-13T05:03:14.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3560999999999999e-05 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978557+0000", 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6729999999999998e-06 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978614+0000", 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6857999999999997e-05 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978633+0000", 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9726000000000001e-05 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.980272+0000", 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016385379999999999 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.980309+0000", 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6572999999999997e-05 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98338 3.f 3:f28d736a:::rbd_data.117cf03415a9.00000000000000b8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2719495079999996, 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021044290000000001, 2024-09-13T05:03:14.770 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98338 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415680+0000", 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415690+0000", 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3910000000000004e-06 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415704+0000", 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4653e-05 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415713+0000", 2024-09-13T05:03:14.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3960000000000006e-06 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415760+0000", 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7222000000000003e-05 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415850+0000", 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0068999999999994e-05 2024-09-13T05:03:14.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415957+0000", 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001066 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416597+0000", 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064032800000000001 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417700+0000", 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001103276 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417746+0000", 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5439999999999999e-05 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417787+0000", 2024-09-13T05:03:14.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0720999999999999e-05 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98341 3.f 3:f15ef2ff:::rbd_data.117cf03415a9.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.2716211050000004, 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021618190000000002, 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98341 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416009+0000", 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416015+0000", 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8429999999999998e-06 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416028+0000", 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2198000000000001e-05 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416035+0000", 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1409999999999999e-06 2024-09-13T05:03:14.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416207+0000", 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017254999999999999 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416310+0000", 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103286 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416371+0000", 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0238000000000002e-05 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416872+0000", 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050110900000000002 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418099+0000", 2024-09-13T05:03:14.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001227271 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418141+0000", 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2089999999999999e-05 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418172+0000", 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1137999999999999e-05 2024-09-13T05:03:14.783 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98738 3.f 3:f4eda464:::rbd_data.117cf03415a9.000000000000007e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8598664600000001, 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018816306000000001, 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:14.784 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98738 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827764+0000", 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827772+0000", 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.585e-06 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827793+0000", 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-05 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827800+0000", 2024-09-13T05:03:14.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8619999999999997e-06 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827849+0000", 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9579000000000001e-05 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827948+0000", 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8382999999999996e-05 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.831301+0000", 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003353698 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846517+0000", 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015215975 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:14.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846526+0000", 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846557+0000", 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1012000000000001e-05 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846581+0000", 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4803999999999998e-05 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.790 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:98738 3.f e66/55 ondisk, result = 0)", 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.8466146769999998, 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055032170000000004, 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:14.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841016+0000", 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841022+0000", 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.711e-06 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841038+0000", 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6419999999999999e-05 2024-09-13T05:03:14.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841050+0000", 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.185e-05 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846506+0000", 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005456079 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846513+0000", 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846520+0000", 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:14.795 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:14.797 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:14.797 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:14.796+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:14.898 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:14.897+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:14.946 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:14.946 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:14.947 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:14.947 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:14.954 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:03:14.959 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:14.999 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:14.998+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:15.034 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:15.077 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:15.077 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:15.077 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:15.077 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:15.077 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:15.087 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:15.129 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:15.128+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:15.161 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:15.161 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:15.178 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:15.178 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:15.194 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:15.194 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:15.200 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:15.200+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:15.210 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:15.211 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:15.227 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:15.227 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:15.237 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.237 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.243 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:15.244 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:15.245 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:15.260 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:15.261 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:15.277 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:15.277 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:15.329 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:15.328+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:15.351 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.351 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.360 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:15.402 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:15.402+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:15.462 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.462 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.471 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:15.503 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:15.503+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:15.574 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.575 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.583 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:15.605 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:15.604+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:15.685 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.685 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.693 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:15.736 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:15.736+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:15.794 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.794 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.803 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:15.844 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:15.843+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:15.907 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:15.907+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:15.945 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:15.945 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:15.954 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:16.008 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:16.007+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:16.069 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.070 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.079 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:16.121 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:16.120+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:16.210 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:16.209+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:16.222 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.222 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.230 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:16.310 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:16.310+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:16.332 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.332 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.341 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:16.426 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:16.425+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:16.483 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.484 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.492 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:16.512 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:16.511+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:16.612 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:16.612+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:16.635 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.636 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.644 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:16.728 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:16.728+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:16.787 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.787 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.796 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:16.837 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:16.836+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:16.915 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:16.915 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:16.916 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:16.915+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:16.925 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:16.972 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:03:17.016 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:17.015+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:17.049 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:17.049 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:17.060 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:17.117 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:17.117+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:17.170 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:17.170 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:17.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:17.218 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:17.217+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:17.288 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:17.319 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:17.318+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:17.328 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:17.328 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:17.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:17.414 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:17.414 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:17.420 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:17.419+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:17.431 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:17.431 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:17.448 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:17.448 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:17.464 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:17.465 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:17.481 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:17.481 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:17.487 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:17.488 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:17.488 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:17.496 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:17.497 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:17.497 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:17.514 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:17.514 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:17.520 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:17.520+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:17.531 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:17.531 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:17.606 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:17.606 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242650222999998, 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242627179999999, 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.611 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:03:17.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242609848000001, 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:03:17.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:03:17.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242593304000003, 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.620 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:03:17.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.624 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242576178999997, 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:17.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:03:17.628 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242559950999997, 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:03:17.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:03:17.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:03:17.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242543140000002, 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.634 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:17.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:03:17.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242526521999999, 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:03:17.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242509943999998, 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.643 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.647 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242473590000003, 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:17.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:03:17.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242454645999999, 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.652 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:03:17.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.656 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242438012000001, 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:17.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:03:17.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242412010999999, 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:03:17.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:03:17.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242380072000003, 2024-09-13T05:03:17.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:03:17.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:03:17.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242361285000001, 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:03:17.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242345165000003, 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:03:17.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242323104, 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.680 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:17.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:03:17.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.683 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:03:17.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.242306139, 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.685 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:03:17.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:03:17.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.930271740000002, 2024-09-13T05:03:17.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:03:17.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 39.930252631999998, 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:17.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:17.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:17.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:17.697 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:17.698 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:17.700 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:17.700 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:17.699+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:17.801 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:17.800+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:17.807 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:17.807 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:17.807 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:17.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:17.902 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:17.901+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:17.922 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:17.922 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:17.922 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:17.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:17.967 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:03:18.002 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:18.002+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:18.082 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:18.082 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:18.082 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-13T05:03:18.082 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.321557 2024-09-13T05:03:18.082 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0.3215573054805978 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:10089 3.9 e29/19 3:9507b4c0:::rbd_data.117c6c62fa22.0000000000000006:head v 29'65, mlcod=28'62)", 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 59.108439783000001, 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0018819710000000001, 2024-09-13T05:03:18.088 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978064+0000", 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978063+0000", 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978072+0000", 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7120000000000006e-06 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978087+0000", 2024-09-13T05:03:18.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5043e-05 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978096+0000", 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8449999999999992e-06 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978161+0000", 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5377999999999996e-05 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.978183+0000", 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1659e-05 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979911+0000", 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017284690000000001 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:18.979946+0000", 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.455e-05 2024-09-13T05:03:18.092 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.283055728000001, 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:03:18.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.097 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 46.141986694000003, 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:03:18.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:18.102 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.909490370999997, 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:18.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:03:18.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40648 3.4 e37/19 3:235438c2:::rbd_data.117c4a06f7bc.00000000000001c8:head v 37'223, mlcod=37'219)", 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.909169493999997, 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833271, 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177334+0000", 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177333+0000", 2024-09-13T05:03:18.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177340+0000", 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9759999999999998e-06 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177352+0000", 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1326e-05 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177359+0000", 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9820000000000002e-06 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177402+0000", 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2774000000000001e-05 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177420+0000", 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8689e-05 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179148+0000", 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017279089999999999 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179167+0000", 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9338999999999999e-05 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.111 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 45.907081537000003, 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:03:18.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:41537 3.9 e38/19 3:95945a69:::rbd_data.117cfc53257b.000000000000006a:head v 38'197, mlcod=38'195)", 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.812772361999997, 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001834647, 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273731+0000", 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.273730+0000", 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274013+0000", 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00028360400000000002 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274033+0000", 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9408000000000001e-05 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274042+0000", 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9539999999999993e-06 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274103+0000", 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1339000000000003e-05 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.274127+0000", 2024-09-13T05:03:18.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3815000000000002e-05 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275523+0000", 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013960979999999999 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:33.275566+0000", 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3117000000000001e-05 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.507633900999998, 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:03:18.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52876 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'388, mlcod=44'385)", 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.748320499000002, 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922389, 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.125 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338182+0000", 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338190+0000", 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.345e-06 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338204+0000", 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4399999999999999e-05 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.127 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338212+0000", 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3880000000000002e-06 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338273+0000", 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1030999999999999e-05 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338293+0000", 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472e-05 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340073+0000", 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017803719999999999 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340106+0000", 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2186e-05 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.129 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52897 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'391, mlcod=44'388)", 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.742918168000003, 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001898648, 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343586+0000", 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343585+0000", 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343592+0000", 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9110000000000004e-06 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343602+0000", 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0701e-05 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343609+0000", 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9099999999999999e-06 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343673+0000", 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4048999999999996e-05 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.343692+0000", 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8946000000000001e-05 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345449+0000", 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017568970000000001 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.345484+0000", 2024-09-13T05:03:18.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5033999999999997e-05 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.728519827, 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:03:18.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:03:18.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:18.138 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.727232825000002, 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:03:18.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.142 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52997 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'399, mlcod=44'395)", 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 37.726036632000003, 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001838309, 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:03:18.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360467+0000", 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360466+0000", 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360473+0000", 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6150000000000002e-06 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360484+0000", 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1549000000000001e-05 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360495+0000", 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0521e-05 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361021+0000", 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000526035 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361040+0000", 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8482e-05 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362269+0000", 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0012290960000000001 2024-09-13T05:03:18.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.362305+0000", 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6733000000000002e-05 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83276 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'726, mlcod=59'724)", 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.110962950000001, 2024-09-13T05:03:18.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039477289999999996, 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975540+0000", 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976050+0000", 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050985799999999997 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976069+0000", 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8393000000000002e-05 2024-09-13T05:03:18.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976098+0000", 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9386e-05 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976193+0000", 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4728999999999994e-05 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976217+0000", 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979453+0000", 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032362860000000001 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979488+0000", 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5629000000000001e-05 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.151 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83277 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'728, mlcod=59'724)", 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.978527+0000", 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 17.107977040000002, 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001781955, 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978527+0000", 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978527+0000", 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978525+0000", 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978535+0000", 2024-09-13T05:03:18.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1779999999999992e-06 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978548+0000", 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3560999999999999e-05 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978557+0000", 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6729999999999998e-06 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978614+0000", 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6857999999999997e-05 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.978633+0000", 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9726000000000001e-05 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.980272+0000", 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016385379999999999 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.980309+0000", 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6572999999999997e-05 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98338 3.f 3:f28d736a:::rbd_data.117cf03415a9.00000000000000b8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6708215699999993, 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021044290000000001, 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.156 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98338 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415680+0000", 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415690+0000", 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3910000000000004e-06 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415704+0000", 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4653e-05 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415713+0000", 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3960000000000006e-06 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415760+0000", 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7222000000000003e-05 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415850+0000", 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0068999999999994e-05 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415957+0000", 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001066 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416597+0000", 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064032800000000001 2024-09-13T05:03:18.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417700+0000", 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001103276 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417746+0000", 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5439999999999999e-05 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417787+0000", 2024-09-13T05:03:18.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0720999999999999e-05 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98341 3.f 3:f15ef2ff:::rbd_data.117cf03415a9.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.6704931670000001, 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021618190000000002, 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.162 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98341 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:18.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416009+0000", 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416015+0000", 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8429999999999998e-06 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416028+0000", 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2198000000000001e-05 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416035+0000", 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1409999999999999e-06 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416207+0000", 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017254999999999999 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416310+0000", 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103286 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416371+0000", 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0238000000000002e-05 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416872+0000", 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050110900000000002 2024-09-13T05:03:18.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418099+0000", 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001227271 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418141+0000", 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2089999999999999e-05 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418172+0000", 2024-09-13T05:03:18.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1137999999999999e-05 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98738 3.f 3:f4eda464:::rbd_data.117cf03415a9.000000000000007e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2587385219999998, 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018816306000000001, 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98738 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827764+0000", 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827772+0000", 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.585e-06 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827793+0000", 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-05 2024-09-13T05:03:18.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827800+0000", 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8619999999999997e-06 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827849+0000", 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9579000000000001e-05 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827948+0000", 2024-09-13T05:03:18.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8382999999999996e-05 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.831301+0000", 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003353698 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846517+0000", 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015215975 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846526+0000", 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846557+0000", 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1012000000000001e-05 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846581+0000", 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4803999999999998e-05 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.173 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:98738 3.f e66/55 ondisk, result = 0)", 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.2454867390000004, 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055032170000000004, 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:18.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841016+0000", 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841022+0000", 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.711e-06 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841038+0000", 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6419999999999999e-05 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841050+0000", 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.185e-05 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.176 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846506+0000", 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005456079 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846513+0000", 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846520+0000", 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.177 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1784 3.4 e74/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:23 v 74'953, mlcod=73'950)", 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.112305266, 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001950465, 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:18.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974203+0000", 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5199999999999999e-06 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974215+0000", 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2211e-05 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974224+0000", 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1870000000000003e-06 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974291+0000", 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7115999999999999e-05 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974311+0000", 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.022e-05 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.976116+0000", 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001804976 2024-09-13T05:03:18.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.976149+0000", 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2472999999999999e-05 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.182 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:18.184 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:18.184 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:18.183+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:18.284 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:18.284+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:18.331 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:18.331 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:18.340 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:18.385 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:18.384+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:18.446 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:18.446 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:18.447 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:18.447 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:18.455 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:18.499 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:18.499+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:18.563 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.044454199, 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.564 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:18.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:18.570 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.044282894000006, 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:18.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:18.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:18.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:18.576 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.117342651000001, 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:18.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.580 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.117332871999999, 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.581 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:18.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.585 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.117318833999999, 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.586 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:18.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.117292668000001, 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.590 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:18.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.012414216, 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:18.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.010160181, 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:18.599 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:18.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.603 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.010143806999999, 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:18.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.605 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:18.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:18.608 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.008313716, 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:18.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.008288592, 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.613 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:18.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.008278189999999, 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:18.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.005959824000001, 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.629 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.005935241, 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:18.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:18.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.005852277999999, 2024-09-13T05:03:18.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:18.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.638 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.005831697000001, 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:18.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:18.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:18.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:18.643 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.005815123000001, 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:18.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.005798125999998, 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.648 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:18.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.652 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.004501440999999, 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:18.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 22.004484557000001, 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:18.657 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:18.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:18.660 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:18.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:18.662 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:18.663 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:18.664 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:18.663+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:18.764 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:18.763+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:18.777 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:18.777 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:18.778 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:18.779 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:18.786 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:18.864 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:18.864+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:18.890 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:18.890 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:18.890 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:18.891 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:18.891 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:18.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:18.966 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:18.965+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:18.970 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.7 primary-affinity to 0.321557 (8210732) 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.810064892, 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:03:19.054 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:03:19.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:03:19.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.810048637000001, 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.059 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.063 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.810031547999998, 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:03:19.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:03:19.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.810014680999998, 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:19.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:03:19.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809965382999998, 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:03:19.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:19.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:03:19.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.078 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809942114000002, 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:03:19.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:03:19.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:19.083 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.80992638, 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:19.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:03:19.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:03:19.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809903400000003, 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:03:19.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809854334000001, 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:19.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:03:19.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.097 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809733018999999, 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:03:19.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:03:19.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809677805, 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.103 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:03:19.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:03:19.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809657516000001, 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.108 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:19.112 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809640334999997, 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:19.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:03:19.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809552824999997, 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.117 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:19.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:03:19.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809533577000003, 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.122 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:03:19.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809472462000002, 2024-09-13T05:03:19.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:03:19.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809455286999999, 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.131 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:03:19.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.135 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809438876999998, 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.136 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:03:19.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.140 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809417893000003, 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:03:19.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:03:19.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.809400469000003, 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.146 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:19.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:03:19.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.150 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:19.152 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:19.152 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:19.151+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:19.252 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:19.252+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:101290 3.d 3.a96ecc8d (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e75)", 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:03:19.043705+0000", 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 0.211322649, 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.21134074, 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "waiting for peered", 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.255 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 101290 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043705+0000", 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043705+0000", 2024-09-13T05:03:19.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043704+0000", 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043707+0000", 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.399e-06 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043719+0000", 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1612e-05 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043725+0000", 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0909999999999998e-06 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043762+0000", 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7299999999999999e-05 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for peered", 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:03:19.043771+0000", 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7409999999999998e-06 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 1 2024-09-13T05:03:19.259 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:19.263 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:19.353 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:19.353+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:19.406 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:19.406 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:19.406 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:19.406 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:19.406 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:19.414 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:19.454 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:19.454+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:19.517 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:19.517 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899381755, 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.519 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:19.520 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.521 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.522 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899363176000001, 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.523 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:19.524 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.525 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.526 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.527 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899338382, 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.528 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.529 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:19.530 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.531 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899319026000001, 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.532 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.533 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.534 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:19.535 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.536 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.89930034, 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.537 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:19.538 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.539 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:19.540 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899275271, 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:19.541 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.542 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.543 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:19.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899252896, 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899232078000001, 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.550 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.553 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.554 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899206774, 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:19.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:19.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899188048999999, 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.563 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899169988000001, 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.567 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899137756999998, 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:19.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.572 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899118597000001, 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:19.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.576 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899101495, 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.577 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:19.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899075197999998, 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.581 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:19.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899049350999999, 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.586 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:19.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:19.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899032499, 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:19.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:19.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:19.594 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899016139, 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:19.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.598 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.899000014999999, 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.599 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:19.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:19.603 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 24.898978785000001, 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:19.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:19.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:19.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:19.608 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:19.608 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:19.610 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:19.611 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:19.610+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:19.710 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:19.710+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:19.713 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:19.713 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:19.713 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:19.713 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:19.722 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:19.745 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:19.745 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:19.761 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:19.762 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:19.777 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:19.777 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:19.793 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:19.793 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:19.809 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:19.810 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:19.810 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:19.810+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:19.826 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:19.826 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:19.842 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:19.842 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:19.858 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:19.859 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:19.865 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:19.869 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:19.874 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:19.916 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:19.915+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:19.985 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:03:20.019 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.020 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:20.029 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:20.029+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:20.114 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:20.113+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:20.131 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.131 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.140 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:20.215 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:20.214+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:20.242 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.242 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.251 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:20.315 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:20.315+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:20.393 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.393 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.401 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:20.416 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:20.415+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:20.509 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.509 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.516 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:20.516+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:20.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:20.617 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:20.616+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:20.624 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.624 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.632 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:20.718 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:20.718+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:20.738 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.739 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.747 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:20.820 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:20.818+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:20.854 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.854 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.863 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:20.920 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:20.920+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:20.970 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:20.971 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:20.975 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:03:20.983 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:21.021 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:21.020+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:21.122 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:21.121+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:21.150 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:21.150 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:21.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:21.243 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:21.243+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:21.307 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:21.307 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:21.315 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:21.357 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:21.356+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:21.425 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:21.425+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:21.463 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:21.463 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:21.471 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:21.556 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:21.555+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:21.619 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:21.619 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:21.628 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:21.628 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:21.627+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:21.729 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:21.729+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:21.775 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:21.775 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:21.784 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:21.869 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:21.868+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:21.871 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:21.931 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:21.930+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:21.931 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:21.931 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:21.941 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:22.004 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:22.004 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:22.023 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:22.023 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:22.031 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:22.031+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:22.041 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:22.041 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:22.060 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:22.060 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:22.078 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:22.078 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:22.095 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:22.096 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.581681345000007, 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:22.101 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:22.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:22.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 67.581510039999998, 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:22.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:22.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.654569797000001, 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:22.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:22.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:22.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.654560018000002, 2024-09-13T05:03:22.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:22.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.654545980000002, 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:22.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.654519814, 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:22.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.549641361999999, 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.131 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.132 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:22.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.135 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.547387326999999, 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.138 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:22.139 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.547370953000001, 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.140 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:22.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:22.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:22.144 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.545540861999999, 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:22.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.545515737999999, 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:22.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:22.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.545505336000002, 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.153 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:22.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:22.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:22.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.543186970000001, 2024-09-13T05:03:22.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:22.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.543162386999999, 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:22.162 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:22.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.165 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.543079423999998, 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.167 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.171 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.543058843000001, 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.543042269000001, 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:22.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:22.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.543025272000001, 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.181 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:22.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:22.185 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.541728587000001, 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:22.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:22.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 25.541711703000001, 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.192 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:22.193 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:22.194 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:22.194 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:22.195 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:22.196 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:22.196 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:22.196 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:22.197 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:22.197 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:22.195+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:22.296 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:22.295+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:22.300 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:22.300 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:22.300 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:22.300 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:22.309 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:22.397 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:22.396+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:22.451 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:22.451 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:22.451 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:22.460 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:22.497 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:22.497+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:22.598 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:22.598+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:22.603 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:22.603 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359879644000003, 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359863388999997, 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359846300000001, 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:03:22.614 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:22.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:03:22.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:03:22.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:03:22.619 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359829433000002, 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:03:22.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:03:22.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:22.625 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359780135000001, 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:03:22.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.630 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359756865999998, 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:22.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:03:22.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:03:22.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359741132000003, 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:03:22.635 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359718151999999, 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:03:22.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359669085999997, 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.646 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:03:22.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359547771000003, 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.651 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:03:22.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.655 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359492557000003, 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:03:22.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.660 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359472267999998, 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.661 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:03:22.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.665 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359455087000001, 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:03:22.666 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:03:22.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:03:22.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359367577, 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.672 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:03:22.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.676 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359348328999999, 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:03:22.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.679 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:03:22.681 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:22.682 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359287213999998, 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.683 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.687 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359270039000002, 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.688 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:03:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359253629000001, 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:03:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:03:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359232644999999, 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:03:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.702 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:03:22.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:03:22.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:03:22.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:22.704 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 55.359215220999999, 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.705 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:03:22.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:22.709 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:22.711 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:22.712 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:22.711+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:22.812 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:22.812+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:22.813 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:22.813 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:22.813 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:22.813 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:22.822 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:22.914 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:22.913+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:22.924 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:22.924 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:22.924 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:22.933 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:22.992 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:03:23.036 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:23.036+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:23.090 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:23.090 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:23.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:23.090 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:23.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.090 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471875287, 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:23.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:23.095 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471856708000001, 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:23.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:23.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:23.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471831913999999, 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.100 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.104 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471812558, 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:23.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:23.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471793871999999, 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.109 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:23.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:23.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:23.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471768803, 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:23.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:23.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471746427999999, 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:23.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:23.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.124 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.47172561, 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:23.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:23.129 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471700305999999, 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:23.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:23.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:23.134 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471681580999999, 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.47166352, 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:23.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:23.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:23.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471631289000001, 2024-09-13T05:03:23.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:23.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:23.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:23.151 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471612129, 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.152 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:23.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471595026999999, 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.157 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:23.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471568730000001, 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:23.161 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:23.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471542883000001, 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:23.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471526031, 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.170 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:23.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:23.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471509671, 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:23.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471493547000001, 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:23.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.471472317, 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:23.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.188 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:23.190 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:23.190 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:23.189+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:23.291 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:23.290+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:23.293 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:23.293 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:23.293 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:23.293 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:23.301 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:23.392 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:23.391+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:23.403 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:23.403 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:23.403 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:23.412 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:23.495 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:23.495+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70864 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:23.556 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.975287298000001, 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038688958000000002, 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70864 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579270+0000", 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.491e-06 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579283+0000", 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2653999999999999e-05 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579289+0000", 2024-09-13T05:03:23.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.614870+0000", 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035581387999999999 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615507+0000", 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00063753200000000001 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617957+0000", 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024498380000000002 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70866 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2596864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.975096939, 2024-09-13T05:03:23.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036734241000000001, 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70866 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579458+0000", 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579462+0000", 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579471+0000", 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9639999999999992e-06 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579477+0000", 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0000000000000002e-06 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615818+0000", 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036340823000000001 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616117+0000", 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029901500000000002 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616193+0000", 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5829999999999997e-05 2024-09-13T05:03:23.565 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70869 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.975072701999999, 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036867536999999999, 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70869 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579482+0000", 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579485+0000", 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3329999999999999e-06 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579491+0000", 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1580000000000002e-06 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579496+0000", 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8679999999999998e-06 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616223+0000", 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036727187000000001 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616290+0000", 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7160000000000001e-05 2024-09-13T05:03:23.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616350+0000", 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0248999999999999e-05 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70877 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.974516379000001, 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036468836999999997, 2024-09-13T05:03:23.570 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70877 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580043+0000", 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3989999999999997e-06 2024-09-13T05:03:23.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580052+0000", 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6319999999999997e-06 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580065+0000", 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3288000000000001e-05 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616372+0000", 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036307147999999997 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616447+0000", 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5296999999999994e-05 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616508+0000", 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0751999999999997e-05 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70879 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1130496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.973769536999999, 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035893590000000003, 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70879 2024-09-13T05:03:23.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580785+0000", 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580792+0000", 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9029999999999998e-06 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580804+0000", 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1545999999999999e-05 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580811+0000", 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2609999999999996e-06 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616530+0000", 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035719133 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616604+0000", 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3712000000000002e-05 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616680+0000", 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5957000000000005e-05 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70882 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.973738071, 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036032559999999998, 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70882 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:23.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580817+0000", 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580820+0000", 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5720000000000001e-06 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580826+0000", 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4130000000000003e-06 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580831+0000", 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.865e-06 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616703+0000", 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035871694000000003 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616782+0000", 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9369999999999994e-05 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616850+0000", 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8118000000000001e-05 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.583 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70891 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1089536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.973718978000001, 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036169077000000001, 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70891 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580836+0000", 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580839+0000", 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3760000000000002e-06 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580853+0000", 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4735999999999999e-05 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580859+0000", 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3329999999999996e-06 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616872+0000", 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036013495999999999 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616945+0000", 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2646999999999997e-05 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617006+0000", 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0853000000000002e-05 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70893 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:23.588 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.973685153000002, 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036286807999999997, 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70893 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580873+0000", 2024-09-13T05:03:23.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7930000000000002e-06 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580880+0000", 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9970000000000001e-06 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580885+0000", 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5350000000000004e-06 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:23.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036141860999999997 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617091+0000", 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3730000000000001e-05 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6412e-05 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70895 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1187840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.973574401, 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036346900000000001, 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70895 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.593 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580985+0000", 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4009999999999999e-06 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581000+0000", 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5024999999999999e-05 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581006+0000", 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3080000000000004e-06 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617178+0000", 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036171571999999999 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617328+0000", 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1537000000000002e-05 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71006 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.968972087000001, 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035821812000000001, 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71006 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585583+0000", 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585586+0000", 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585593+0000", 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9210000000000003e-06 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585598+0000", 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0159999999999999e-06 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621279+0000", 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035681069000000003 2024-09-13T05:03:23.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621346+0000", 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7080000000000001e-05 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621405+0000", 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9401999999999998e-05 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.601 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71008 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.968950826, 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035951295000000001, 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71008 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.602 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585604+0000", 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585607+0000", 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.266e-06 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585613+0000", 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585622+0000", 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7590000000000003e-06 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621431+0000", 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035809331 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621495+0000", 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4611999999999997e-05 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621556+0000", 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0591999999999999e-05 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71009 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.968929424999999, 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036076001000000003, 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.606 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71009 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585634+0000", 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.055e-06 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585640+0000", 2024-09-13T05:03:23.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4189999999999999e-06 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585646+0000", 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621580+0000", 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035934440999999998 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621644+0000", 2024-09-13T05:03:23.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3927e-05 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621702+0000", 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7855000000000001e-05 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71011 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.968900275999999, 2024-09-13T05:03:23.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036193110000000001, 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71011 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585658+0000", 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7329999999999999e-06 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585666+0000", 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7280000000000001e-06 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585671+0000", 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-06 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621726+0000", 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036054614999999998 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621790+0000", 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3868000000000006e-05 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621848+0000", 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8913000000000003e-05 2024-09-13T05:03:23.614 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71071 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.96799116, 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035743500999999997, 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.615 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71071 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586566+0000", 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3209999999999999e-06 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586572+0000", 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586578+0000", 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622184+0000", 2024-09-13T05:03:23.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035606024999999999 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622248+0000", 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4691999999999997e-05 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622308+0000", 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9663000000000001e-05 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.619 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71096 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.967016372, 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035817952, 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71096 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.620 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587541+0000", 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587548+0000", 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4679999999999997e-06 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587561+0000", 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2713e-05 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623235+0000", 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035674375000000001 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623300+0000", 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4505999999999995e-05 2024-09-13T05:03:23.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623357+0000", 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7661999999999998e-05 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71097 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.966987306, 2024-09-13T05:03:23.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035944619999999997, 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71097 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587575+0000", 2024-09-13T05:03:23.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587582+0000", 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1160000000000001e-06 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587587+0000", 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0490000000000003e-06 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623383+0000", 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035795897 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623448+0000", 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4604999999999996e-05 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623513+0000", 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5168999999999999e-05 2024-09-13T05:03:23.628 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71115 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.966159696999998, 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036023959000000001, 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.629 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71115 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588395+0000", 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588400+0000", 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588410+0000", 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0076e-05 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588417+0000", 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6549999999999998e-06 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623694+0000", 2024-09-13T05:03:23.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035277159000000002 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623760+0000", 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5968000000000003e-05 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624420+0000", 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066017099999999998 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71117 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.965911303999999, 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035938036, 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71117 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.634 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588643+0000", 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588649+0000", 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5210000000000002e-06 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588660+0000", 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1459e-05 2024-09-13T05:03:23.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588668+0000", 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5669999999999997e-06 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624447+0000", 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035778841999999998 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624516+0000", 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8774000000000003e-05 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624582+0000", 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6818999999999998e-05 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.638 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71119 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.965592137000002, 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035764241000000002, 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71119 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588962+0000", 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:23.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588968+0000", 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5770000000000001e-06 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588980+0000", 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1718e-05 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588987+0000", 2024-09-13T05:03:23.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624607+0000", 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035619545000000002 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624670+0000", 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3664999999999994e-05 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624728+0000", 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7433999999999999e-05 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71120 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1220608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 28.965411402000001, 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035736393999999998, 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:23.643 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71120 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589143+0000", 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589149+0000", 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1329999999999998e-06 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589167+0000", 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8236999999999998e-05 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589174+0000", 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4889999999999998e-06 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624752+0000", 2024-09-13T05:03:23.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035577723999999998 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624813+0000", 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1138999999999999e-05 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624881+0000", 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7489000000000004e-05 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.647 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:23.648 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:23.648 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:23.649 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:23.650 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:23.649+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:23.750 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:23.749+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:23.793 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:23.793 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:23.793 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:23.802 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:23.851 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:23.850+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:23.944 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:23.944 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:23.945 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:23.945 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:23.951 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:23.950+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:23.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:23.987 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:03:23.990 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:23.990 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:23.990 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-09-13T05:03:23.990 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:03:23.990 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:03:23.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-09-13T05:03:24.051 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:24.051+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.874256107000001, 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:24.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:24.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.873327998000001, 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.110 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.872229589, 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:24.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:24.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.866169628000002, 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:24.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:24.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:24.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.125 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.866160633, 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.126 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:24.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.130 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.866152262, 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.131 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.866142240999999, 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:24.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:24.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.866126518000002, 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:24.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.864224348999997, 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:24.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.864216390000003, 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.149 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:24.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.153 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.864208701999999, 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:24.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.864198010000003, 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.158 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:24.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.864189768999999, 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:24.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:24.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.864166107999999, 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.168 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:24.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.862523649000003, 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.173 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:24.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:24.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.862515416000001, 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:24.177 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:24.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.181 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.862292687, 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:24.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:24.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.862280138000003, 2024-09-13T05:03:24.186 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:24.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.860240912000002, 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:24.191 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.192 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:24.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:24.194 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:24.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 59.860231132999999, 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:24.197 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:24.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:24.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:24.203 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:24.204 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:24.206 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:24.206 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T05:03:24.206 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:24.206 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:24.204+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:24.206 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:24.305 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:24.305+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:24.355 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:24.356 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:24.356 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:24.356 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:24.365 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:24.370 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:24.371 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:24.388 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:24.388 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:24.406 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:24.406 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:24.406 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:24.406+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:24.422 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:24.423 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:24.439 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:24.439 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:24.456 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:24.456 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:24.472 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:24.473 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:24.479 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:24.479 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:24.488 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:24.489 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:24.489 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:24.541 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:24.541+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:24.595 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:24.595 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:24.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:24.607 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:24.606+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:24.707 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:24.707+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:24.751 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:24.751 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:24.761 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:24.844 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:24.844+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:24.909 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:24.908+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:24.916 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:24.916 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:24.926 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:25.010 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:25.010+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:25.075 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.076 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.084 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:25.111 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:25.110+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:25.211 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:25.211+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:25.232 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.232 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.241 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:25.325 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:25.325+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:25.388 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.388 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.397 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:25.438 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:25.437+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:25.514 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:25.514+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:25.544 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.545 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.553 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:25.637 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:25.637+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:25.700 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.700 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.709 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:25.750 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:25.750+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:25.817 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:25.817+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:25.855 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.856 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.864 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:25.918 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:25.918+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:25.972 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:25.972 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:25.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:26.007 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:03:26.025 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:26.025+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:26.107 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:26.107 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:26.117 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:26.159 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:26.158+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:26.220 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:26.220+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:26.261 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:26.261 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:26.270 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:26.353 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:26.353+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:26.412 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:26.412 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:26.421 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:26.462 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:26.461+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:26.501 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:26.546 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:26.546+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:26.564 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:26.564 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:26.573 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:26.625 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:26.624+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:26.627 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:26.627 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:26.644 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:26.644 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:26.660 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:26.660 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:26.675 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:26.676 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:26.677 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:26.677 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:26.685 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:26.692 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:26.692 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:26.708 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:26.709 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:26.725 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:26.725 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:26.725 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:26.724+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:26.741 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:26.742 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.790 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171854826000001, 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:26.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171836247000002, 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.795 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:26.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.799 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171811452999997, 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.800 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.801 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:26.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.804 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171792097000001, 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:26.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171773410999997, 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.809 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.810 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:26.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:26.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171748342000001, 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.814 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:26.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171725967, 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:26.818 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:26.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:26.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171705148999997, 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.823 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.824 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:26.825 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.826 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.827 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171679845, 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.828 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.829 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.830 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.831 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171661120000003, 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:26.832 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.833 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:26.834 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.835 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:26.836 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171643058999997, 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.837 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.838 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:26.839 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.840 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.841 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171610827999999, 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.842 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.843 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:26.844 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:26.845 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171591667999998, 2024-09-13T05:03:26.846 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.847 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.849 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.850 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171574565999997, 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:26.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:26.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171548268999999, 2024-09-13T05:03:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:26.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:26.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171522422000002, 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.860 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:26.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.864 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171505570000001, 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:26.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171489209999997, 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.869 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:26.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:26.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171473085999999, 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.874 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:26.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:26.878 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.171451855999997, 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.879 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:26.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:26.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:26.883 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:26.885 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:26.885 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:26.884+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:26.985 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:26.985+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:26.993 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:03:27.038 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:27.038 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:27.038 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:27.038 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:27.048 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:27.091 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:27.091+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:27.153 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:27.153 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:27.154 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:27.162 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:27.202 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:27.202+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:27.288 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:27.288+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70864 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.724592927000003, 2024-09-13T05:03:27.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038688958000000002, 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70864 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579270+0000", 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.491e-06 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579283+0000", 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2653999999999999e-05 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579289+0000", 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.614870+0000", 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035581387999999999 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615507+0000", 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00063753200000000001 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617957+0000", 2024-09-13T05:03:27.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024498380000000002 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70866 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2596864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.724402568000002, 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036734241000000001, 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.310 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70866 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579458+0000", 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579462+0000", 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579471+0000", 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9639999999999992e-06 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579477+0000", 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0000000000000002e-06 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615818+0000", 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036340823000000001 2024-09-13T05:03:27.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616117+0000", 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029901500000000002 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616193+0000", 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5829999999999997e-05 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70869 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.724378330999997, 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036867536999999999, 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.315 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70869 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579482+0000", 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579485+0000", 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3329999999999999e-06 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579491+0000", 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1580000000000002e-06 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579496+0000", 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8679999999999998e-06 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616223+0000", 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036727187000000001 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616290+0000", 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7160000000000001e-05 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616350+0000", 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0248999999999999e-05 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70877 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.723822007999999, 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036468836999999997, 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70877 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:27.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580043+0000", 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3989999999999997e-06 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580052+0000", 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6319999999999997e-06 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580065+0000", 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3288000000000001e-05 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616372+0000", 2024-09-13T05:03:27.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036307147999999997 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616447+0000", 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5296999999999994e-05 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616508+0000", 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0751999999999997e-05 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.323 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70879 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1130496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.723075166000001, 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035893590000000003, 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.324 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70879 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580785+0000", 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580792+0000", 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9029999999999998e-06 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580804+0000", 2024-09-13T05:03:27.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1545999999999999e-05 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580811+0000", 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2609999999999996e-06 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616530+0000", 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035719133 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616604+0000", 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3712000000000002e-05 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616680+0000", 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5957000000000005e-05 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.328 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70882 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.723043699999998, 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036032559999999998, 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70882 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580817+0000", 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580820+0000", 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5720000000000001e-06 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580826+0000", 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4130000000000003e-06 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580831+0000", 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.865e-06 2024-09-13T05:03:27.331 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616703+0000", 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035871694000000003 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616782+0000", 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9369999999999994e-05 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.332 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616850+0000", 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8118000000000001e-05 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70891 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1089536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.723024606999999, 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036169077000000001, 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.333 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70891 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.334 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580836+0000", 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580839+0000", 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3760000000000002e-06 2024-09-13T05:03:27.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580853+0000", 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4735999999999999e-05 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580859+0000", 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3329999999999996e-06 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616872+0000", 2024-09-13T05:03:27.336 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036013495999999999 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616945+0000", 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2646999999999997e-05 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617006+0000", 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0853000000000002e-05 2024-09-13T05:03:27.337 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70893 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.722990781999997, 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036286807999999997, 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.338 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70893 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.339 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580873+0000", 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7930000000000002e-06 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580880+0000", 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9970000000000001e-06 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.340 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580885+0000", 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5350000000000004e-06 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036141860999999997 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617091+0000", 2024-09-13T05:03:27.341 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3730000000000001e-05 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6412e-05 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70895 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1187840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:27.342 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.722880029999999, 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036346900000000001, 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70895 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.343 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:27.344 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580985+0000", 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4009999999999999e-06 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581000+0000", 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5024999999999999e-05 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.345 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581006+0000", 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3080000000000004e-06 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617178+0000", 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036171571999999999 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.346 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617328+0000", 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1537000000000002e-05 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.347 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71006 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.718277716000003, 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035821812000000001, 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71006 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.348 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585583+0000", 2024-09-13T05:03:27.349 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585586+0000", 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585593+0000", 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9210000000000003e-06 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.350 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585598+0000", 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0159999999999999e-06 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621279+0000", 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035681069000000003 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621346+0000", 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7080000000000001e-05 2024-09-13T05:03:27.351 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621405+0000", 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9401999999999998e-05 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71008 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.718256455000002, 2024-09-13T05:03:27.352 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035951295000000001, 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71008 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.353 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585604+0000", 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.354 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585607+0000", 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.266e-06 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585613+0000", 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585622+0000", 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7590000000000003e-06 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.355 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621431+0000", 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035809331 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621495+0000", 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4611999999999997e-05 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621556+0000", 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0591999999999999e-05 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.356 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71009 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.718235053999997, 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036076001000000003, 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71009 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.358 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585634+0000", 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.055e-06 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585640+0000", 2024-09-13T05:03:27.359 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4189999999999999e-06 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585646+0000", 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621580+0000", 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035934440999999998 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.360 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621644+0000", 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3927e-05 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621702+0000", 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7855000000000001e-05 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.361 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71011 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.718205904999998, 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036193110000000001, 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71011 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.362 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.363 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585658+0000", 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7329999999999999e-06 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585666+0000", 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7280000000000001e-06 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.364 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585671+0000", 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-06 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621726+0000", 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036054614999999998 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621790+0000", 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3868000000000006e-05 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.365 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621848+0000", 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8913000000000003e-05 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71071 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.717296789000002, 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035743500999999997, 2024-09-13T05:03:27.366 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71071 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.367 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586566+0000", 2024-09-13T05:03:27.368 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3209999999999999e-06 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586572+0000", 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586578+0000", 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.369 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622184+0000", 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035606024999999999 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622248+0000", 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4691999999999997e-05 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622308+0000", 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9663000000000001e-05 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.370 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71096 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.716322001000002, 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035817952, 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.371 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71096 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.372 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587541+0000", 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.373 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587548+0000", 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4679999999999997e-06 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587561+0000", 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2713e-05 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623235+0000", 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035674375000000001 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.374 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623300+0000", 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4505999999999995e-05 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623357+0000", 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7661999999999998e-05 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.375 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71097 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.716292934999998, 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035944619999999997, 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71097 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.376 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:27.377 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587575+0000", 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587582+0000", 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1160000000000001e-06 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.378 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587587+0000", 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0490000000000003e-06 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623383+0000", 2024-09-13T05:03:27.379 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035795897 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623448+0000", 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4604999999999996e-05 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623513+0000", 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5168999999999999e-05 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.380 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71115 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.715465326, 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036023959000000001, 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71115 2024-09-13T05:03:27.381 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.382 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588395+0000", 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588400+0000", 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588410+0000", 2024-09-13T05:03:27.383 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0076e-05 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588417+0000", 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6549999999999998e-06 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623694+0000", 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035277159000000002 2024-09-13T05:03:27.384 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623760+0000", 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5968000000000003e-05 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624420+0000", 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066017099999999998 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.385 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71117 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.715216933000001, 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035938036, 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71117 2024-09-13T05:03:27.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588643+0000", 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588649+0000", 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5210000000000002e-06 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588660+0000", 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1459e-05 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588668+0000", 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5669999999999997e-06 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624447+0000", 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035778841999999998 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624516+0000", 2024-09-13T05:03:27.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8774000000000003e-05 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624582+0000", 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6818999999999998e-05 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71119 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.714897766, 2024-09-13T05:03:27.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035764241000000002, 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71119 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588962+0000", 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588968+0000", 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5770000000000001e-06 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588980+0000", 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1718e-05 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588987+0000", 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624607+0000", 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035619545000000002 2024-09-13T05:03:27.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624670+0000", 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3664999999999994e-05 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624728+0000", 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7433999999999999e-05 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71120 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1220608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 32.714717030999999, 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035736393999999998, 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71120 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.395 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589143+0000", 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589149+0000", 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1329999999999998e-06 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589167+0000", 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8236999999999998e-05 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589174+0000", 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4889999999999998e-06 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624752+0000", 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035577723999999998 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624813+0000", 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1138999999999999e-05 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624881+0000", 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7489000000000004e-05 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.399 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:27.401 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:27.401 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:27.400+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:27.501 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:27.501+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:27.543 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:27.544 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:27.544 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:27.544 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:27.552 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:27.636 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:27.636+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:27.695 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:27.695 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:27.695 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:27.703 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:27.704 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:27.703+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:27.804 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:27.803+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:27.848 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:27.848 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:27.848 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:27.848 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:27.848 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.618058108, 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.849 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:27.850 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:27.851 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.852 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.617129998999999, 2024-09-13T05:03:27.853 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.854 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.855 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:27.856 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.857 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.616031589999999, 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.858 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.859 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:27.860 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:27.861 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.862 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.863 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.863 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.863 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.609971629, 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.864 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.865 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.866 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:27.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.609962633999999, 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.870 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:27.871 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.872 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.873 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.609954262999999, 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:27.874 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:27.876 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.877 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.878 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.609944241999997, 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:27.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.880 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.881 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:27.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.609928519, 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:27.883 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.884 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.885 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:27.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:27.887 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.608026350000003, 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.888 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:27.889 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.890 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.891 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.892 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.608018391000002, 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.893 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.894 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:27.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.896 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.608010702999998, 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.897 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:27.898 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.899 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:27.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.901 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.608000011000001, 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.902 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:27.903 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.904 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.905 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.607991769999998, 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.906 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.907 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:27.908 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:27.909 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.910 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.607968108999998, 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.911 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.912 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.913 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:27.914 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.606325650000002, 2024-09-13T05:03:27.915 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.916 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.917 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.918 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:27.919 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.606317417, 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.920 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.921 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:27.922 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.923 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:27.924 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.606094687999999, 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:27.925 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.926 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.927 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:27.928 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.606082139000002, 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.929 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:27.930 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.931 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:27.932 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.933 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.604042913000001, 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.934 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.935 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.936 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:27.937 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:27.938 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 63.604033133999998, 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.940 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:27.941 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.942 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:27.943 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:27.944 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.945 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:27.946 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:27.946 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:27.947 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:27.948 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:27.947+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:28.047 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:28.047+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:28.085 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:28.085 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:28.086 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:28.095 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:28.148 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:28.147+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:28.243 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:28.243 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:28.243 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:28.248 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:28.248+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:28.252 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:28.349 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:28.349+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:28.360 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703548079000001, 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:28.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.362 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.364 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703532082000002, 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:03:28.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703515285999998, 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:03:28.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703498312999997, 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.373 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:03:28.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.376 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703481918999998, 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.377 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:03:28.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703464883000002, 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.381 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703431299999998, 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:03:28.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:03:28.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703412170999997, 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:03:28.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.393 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703334302000002, 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:03:28.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:03:28.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703273557000003, 2024-09-13T05:03:28.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:03:28.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:03:28.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.402 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703213343000002, 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:03:28.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:03:28.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:03:28.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703196016, 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.407 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:03:28.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703166844000002, 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.411 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:28.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.413 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.70314716, 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:03:28.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.416 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:03:28.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703129726999997, 2024-09-13T05:03:28.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:28.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703112085000001, 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:28.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:03:28.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703077909000001, 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.428 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:28.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703052559, 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.432 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.434 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703034885999998, 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:28.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.703010681999999, 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:03:28.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:03:28.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.445 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.445 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.445 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.445 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:28.446 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:28.450 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:28.450+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:28.551 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:28.550+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:28.553 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:28.553 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:28.553 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:28.562 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:28.651 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:28.651+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:28.709 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:28.709 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:28.709 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:28.718 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:28.753 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:28.762 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:28.762+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:28.827 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:28.827 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463962219000003, 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.828 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:03:28.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.832 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463939175999997, 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463921843999998, 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:03:28.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:03:28.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.4639053, 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:03:28.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:03:28.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463888175000001, 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:03:28.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:03:28.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463871947000001, 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:03:28.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:03:28.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463855135999999, 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.855 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:03:28.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:03:28.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463838518000003, 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.859 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463821940000003, 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.863 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:28.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.867 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463785586, 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:03:28.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:03:28.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463766642000003, 2024-09-13T05:03:28.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:28.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463750007999998, 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.877 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:28.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463724007000003, 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.881 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:03:28.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:03:28.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463692068, 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.886 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:03:28.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:03:28.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:03:28.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:28.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463673280999998, 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:03:28.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463657161, 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:03:28.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:03:28.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463635099999998, 2024-09-13T05:03:28.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:03:28.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.463618134999997, 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.151583735999999, 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.909 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:03:28.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 51.151564628000003, 2024-09-13T05:03:28.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:28.917 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:28.919 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:28.920 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:28.920 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:28.920 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:28.921 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:28.921 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:28.921 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:28.921 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:28.918+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:28.933 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:28.933 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:28.950 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:28.950 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:28.967 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:28.967 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:28.984 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:28.985 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:29.003 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:29.003 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:29.013 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:03:29.020 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:29.019+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:29.049 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.049 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:29.121 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:29.120+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:29.207 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:29.207 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:29.207 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-13T05:03:29.207 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:03:29.207 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:03:29.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-13T05:03:29.208 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.208 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.217 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:29.261 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:29.261+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:29.314 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:29.314 INFO:teuthology.orchestra.run.smithi110.stdout: "success": "" 2024-09-13T05:03:29.314 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:29.323 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:29.322+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:29.324 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.324 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.333 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:29.424 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:29.423+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:29.481 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.481 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.490 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:29.534 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:29.534+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:29.597 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.597 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.605 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:29.647 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:29.645+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:29.726 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:29.726+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:29.752 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.752 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.760 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:29.827 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:29.826+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:29.865 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:29.865 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:29.873 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:29.878 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:03:29.928 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:29.926+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:30.021 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.021 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.028 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:30.027+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:30.030 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:30.129 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:30.128+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:30.131 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.131 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.140 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:30.230 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:30.230+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:30.242 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.242 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.251 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:30.331 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:30.330+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:30.393 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.393 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.402 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:30.445 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:30.445+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:30.504 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.504 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.512 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:30.554 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:30.553+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:30.633 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:30.633+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:30.655 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.655 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.664 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:30.747 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:30.747+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:30.805 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.805 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.814 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:30.835 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:30.834+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:30.935 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:30.934+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:30.966 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:30.967 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:30.976 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:31.017 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:31.036 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:31.035+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:31.079 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:31.080 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:31.088 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:31.142 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:31.142 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:31.159 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:31.159 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:31.172 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:31.172+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:31.175 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:31.175 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:31.190 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:31.191 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:31.208 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:31.208 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:31.225 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:31.226 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:31.233 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:31.233 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:31.237 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:31.237+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:31.242 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:31.243 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:31.243 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:31.259 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:31.259 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:31.339 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:31.338+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:31.385 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:31.385 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.155006985, 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.386 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.387 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:31.388 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:31.389 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:31.390 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.154078876, 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.392 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.393 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:31.394 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.152980467000006, 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.395 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.396 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:31.397 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:31.398 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.146920506000001, 2024-09-13T05:03:31.399 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.400 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:31.401 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.402 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.403 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:31.404 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.146911510999999, 2024-09-13T05:03:31.405 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.406 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.407 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.408 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:31.409 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.146903140000006, 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.410 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.411 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.412 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.413 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.414 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.146893118999998, 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.415 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:31.416 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.417 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.418 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.146877395999994, 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.419 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.420 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.421 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.422 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.423 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.144975227000003, 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:31.424 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.425 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.426 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:31.427 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.144967268000002, 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.428 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.429 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:31.430 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.431 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:31.432 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.144959580000005, 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.433 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:31.434 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.435 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.436 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.144948888000002, 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.437 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:31.438 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.439 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.440 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.144940646999999, 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:31.441 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.442 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:31.443 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.444 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:31.445 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.144916985999998, 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.446 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:31.447 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.448 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:31.449 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.450 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.143274527000003, 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.451 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:31.452 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.453 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:31.454 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.143266294, 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:31.455 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.456 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:31.457 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.458 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.459 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.143043564999999, 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.460 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.461 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:31.462 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.463 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.464 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.143031015999995, 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.465 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.466 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:31.467 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.468 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.140991790000001, 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:31.469 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:31.470 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.471 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.472 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:31.473 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.474 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 67.140982011000006, 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:31.475 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:31.476 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:31.477 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:31.478 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:31.479 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.480 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:31.481 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:31.481 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:31.482 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:31.483 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:31.481+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:31.583 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:31.582+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:31.591 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:31.592 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:31.592 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:31.600 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:31.684 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:31.683+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:31.707 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:31.708 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:31.708 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:31.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:31.785 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:31.785+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:31.865 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.865 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208599913, 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:31.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:03:31.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208583916000002, 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.870 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208567119999998, 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:03:31.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208550146999997, 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:03:31.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208533752999998, 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:03:31.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:03:31.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208516717000002, 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.887 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:03:31.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:03:31.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:31.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208483133999998, 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:31.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.895 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208464005000003, 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.896 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.897 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.898 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.899 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208386136000001, 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.900 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.901 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:03:31.902 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.903 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208325391000002, 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.904 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.905 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.906 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.907 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208265177000001, 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.908 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.909 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.910 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:03:31.911 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.912 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208247849999999, 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.913 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.914 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:03:31.915 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.916 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208218678000001, 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.917 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.918 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:03:31.919 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.920 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208198994, 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.921 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.922 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.923 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.924 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208181561000004, 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.925 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.926 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.927 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:03:31.928 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208163919, 2024-09-13T05:03:31.929 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.930 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:03:31.931 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.932 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.933 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208129743000001, 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.934 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.935 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:03:31.936 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.937 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208104392999999, 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.938 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.939 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:03:31.940 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.941 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208086719999997, 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.942 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.943 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.944 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.945 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.208062515999998, 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:31.946 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.947 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.948 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:03:31.949 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:31.950 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:31.950 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:31.950 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:03:31.951 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:31.950+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:31.952 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:32.052 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:32.051+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:32.102 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:32.103 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:32.103 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:32.112 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:32.153 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:32.152+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:32.254 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:32.252+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:32.259 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:32.259 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:32.259 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:32.259 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:32.259 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:32.268 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:32.354 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:32.353+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:32.417 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:32.417 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053822134000001, 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.418 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.420 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:03:32.421 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:03:32.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:32.423 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053799091000002, 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.424 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.425 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.426 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.427 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.428 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053781759000003, 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.429 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:03:32.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053765214999999, 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:03:32.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.435 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:03:32.437 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053748089999999, 2024-09-13T05:03:32.438 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:32.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.441 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:03:32.442 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053731861999999, 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:03:32.443 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053715050999998, 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.449 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:32.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.453 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053698433000001, 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.454 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:32.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:03:32.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053681855000001, 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:32.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:03:32.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.463 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053645500999998, 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:03:32.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:03:32.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053626557000001, 2024-09-13T05:03:32.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.472 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053609923000003, 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.473 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:32.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:03:32.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:03:32.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053583922000001, 2024-09-13T05:03:32.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.483 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053551982999998, 2024-09-13T05:03:32.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053533195999997, 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:03:32.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:32.494 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053517075999999, 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053495015000003, 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.500 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.053478050000002, 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:03:32.504 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:32.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:03:32.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.741443650999997, 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:32.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:03:32.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 54.741424543000001, 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:03:32.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:32.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:03:32.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:03:32.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.519 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:32.521 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:32.522 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:32.520+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:32.621 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:32.621+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:32.670 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:32.670 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:32.670 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:32.680 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:32.723 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:32.722+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:32.786 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:32.786 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:32.786 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:32.787 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:32.787 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:32.795 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:32.824 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:32.823+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:32.893 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:03:32.924 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:32.923+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:32.959 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 65.153663249000004, 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.960 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.961 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.962 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.963 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 61.012594215, 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:03:32.964 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.965 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.966 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.967 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:03:32.968 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.780097892000001, 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.969 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.970 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.971 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:03:32.972 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.973 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40659 3.4 e37/19 3:210cb3d1:::rbd_data.117c4a06f7bc.0000000000000222:head v 37'224, mlcod=37'220)", 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 60.777689058, 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0019697859999999998, 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.974 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179422+0000", 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179421+0000", 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179428+0000", 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3729999999999998e-06 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.975 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179442+0000", 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4109e-05 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179449+0000", 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179498+0000", 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9066000000000001e-05 2024-09-13T05:03:32.976 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179521+0000", 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2991999999999999e-05 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181355+0000", 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001833912 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.181392+0000", 2024-09-13T05:03:32.977 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6721999999999998e-05 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.378241422000002, 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.978 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.979 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:03:32.980 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.981 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:03:32.982 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52876 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'388, mlcod=44'385)", 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.618928019999998, 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001922389, 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.983 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338183+0000", 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338182+0000", 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:32.984 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338190+0000", 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.345e-06 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338204+0000", 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4399999999999999e-05 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338212+0000", 2024-09-13T05:03:32.985 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3880000000000002e-06 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338273+0000", 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1030999999999999e-05 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.338293+0000", 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0472e-05 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.986 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340073+0000", 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017803719999999999 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.340106+0000", 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2186e-05 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:32.987 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.599127348000003, 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.988 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.989 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:03:32.990 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.991 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:03:32.992 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 52.597840345999998, 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.993 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:03:32.994 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.995 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:03:32.996 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83276 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'726, mlcod=59'724)", 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:32.997 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 31.981570471000001, 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039477289999999996, 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.998 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975540+0000", 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976050+0000", 2024-09-13T05:03:32.999 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050985799999999997 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976069+0000", 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8393000000000002e-05 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976098+0000", 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9386e-05 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.000 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976193+0000", 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4728999999999994e-05 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976217+0000", 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979453+0000", 2024-09-13T05:03:33.001 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032362860000000001 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979488+0000", 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5629000000000001e-05 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98338 3.f 3:f28d736a:::rbd_data.117cf03415a9.00000000000000b8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.541429091000001, 2024-09-13T05:03:33.002 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021044290000000001, 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98338 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.003 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415680+0000", 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.004 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415690+0000", 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3910000000000004e-06 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415704+0000", 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4653e-05 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415713+0000", 2024-09-13T05:03:33.005 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3960000000000006e-06 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415760+0000", 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7222000000000003e-05 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415850+0000", 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0068999999999994e-05 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415957+0000", 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001066 2024-09-13T05:03:33.006 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416597+0000", 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064032800000000001 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417700+0000", 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001103276 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417746+0000", 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5439999999999999e-05 2024-09-13T05:03:33.007 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417787+0000", 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0720999999999999e-05 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98341 3.f 3:f15ef2ff:::rbd_data.117cf03415a9.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.541100688, 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021618190000000002, 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.008 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98341 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.009 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416009+0000", 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416015+0000", 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8429999999999998e-06 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.010 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416028+0000", 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2198000000000001e-05 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416035+0000", 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1409999999999999e-06 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416207+0000", 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017254999999999999 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.011 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416310+0000", 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103286 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416371+0000", 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0238000000000002e-05 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416872+0000", 2024-09-13T05:03:33.012 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050110900000000002 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418099+0000", 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001227271 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418141+0000", 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2089999999999999e-05 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.013 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418172+0000", 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1137999999999999e-05 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98738 3.f 3:f4eda464:::rbd_data.117cf03415a9.000000000000007e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.129346043000002, 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018816306000000001, 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.014 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98738 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.015 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827764+0000", 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827772+0000", 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.585e-06 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827793+0000", 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-05 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.016 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827800+0000", 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8619999999999997e-06 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827849+0000", 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9579000000000001e-05 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827948+0000", 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8382999999999996e-05 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.017 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.831301+0000", 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003353698 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846517+0000", 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015215975 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846526+0000", 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-13T05:03:33.018 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846557+0000", 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1012000000000001e-05 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846581+0000", 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4803999999999998e-05 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.019 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:98738 3.f e66/55 ondisk, result = 0)", 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.116094260000001, 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055032170000000004, 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.020 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841016+0000", 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841022+0000", 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.711e-06 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.021 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841038+0000", 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6419999999999999e-05 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841050+0000", 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.185e-05 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846506+0000", 2024-09-13T05:03:33.022 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005456079 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846513+0000", 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846520+0000", 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.023 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1784 3.4 e74/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:23 v 74'953, mlcod=73'950)", 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 14.982912787, 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001950465, 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:33.024 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974198+0000", 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974203+0000", 2024-09-13T05:03:33.025 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5199999999999999e-06 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974215+0000", 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2211e-05 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974224+0000", 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.1870000000000003e-06 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.026 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974291+0000", 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7115999999999999e-05 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.974311+0000", 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.022e-05 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.976116+0000", 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001804976 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:17.976149+0000", 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2472999999999999e-05 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.4.0:2844 3.9 e79/19 3:9408067b:::rbd_data.117c394b9965.0000000000000074:1f v 79'410, mlcod=77'407)", 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.9738710130000001, 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002066876, 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.028 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983249+0000", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4369999999999996e-06 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983260+0000", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0465999999999999e-05 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983270+0000", 2024-09-13T05:03:33.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0268999999999999e-05 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983668+0000", 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039786299999999998 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983694+0000", 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6500999999999999e-05 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985234+0000", 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001539758 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985307+0000", 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3113000000000003e-05 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.032 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.4.0:2845 3.9 e79/19 3:9799fd9b:::rbd_data.117c394b9965.000000000000001e:1f v 79'412, mlcod=77'407)", 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.9734980740000001, 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002201724, 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.033 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983612+0000", 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983621+0000", 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7639999999999994e-06 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983642+0000", 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1631e-05 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983649+0000", 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6139999999999997e-06 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.984376+0000", 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072730199999999998 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.984393+0000", 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6738000000000001e-05 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985760+0000", 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013673369999999999 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985815+0000", 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4478000000000003e-05 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102158 3.f 3:f4db7252:::rbd_data.117c394b9965.0000000000000010:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T05:03:33.037 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.899967384, 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002237228, 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102158 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057141+0000", 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057155+0000", 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3583e-05 2024-09-13T05:03:33.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057176+0000", 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0815999999999999e-05 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057187+0000", 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0841e-05 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057255+0000", 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8226999999999996e-05 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057343+0000", 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8294999999999994e-05 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057459+0000", 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115439 2024-09-13T05:03:33.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.058518+0000", 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001059286 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059231+0000", 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071358399999999996 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059301+0000", 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9479000000000003e-05 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059381+0000", 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0153000000000003e-05 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1861 3.4 e83/19 3:249fe31d:::rbd_data.117cc3f2292.0000000000000064:1b v 83'980, mlcod=77'977)", 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:33.043 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.9556267350000001, 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022663900000000001, 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:33.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001490+0000", 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7749999999999998e-06 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001497+0000", 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3069999999999997e-06 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001504+0000", 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2819999999999999e-06 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001592+0000", 2024-09-13T05:03:33.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8015000000000003e-05 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001614+0000", 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1163e-05 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.003705+0000", 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002091337 2024-09-13T05:03:33.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.003751+0000", 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5890999999999998e-05 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102933 3.f 3:f3a9a226:::rbd_data.117cc3f2292.000000000000000a:head [delete] snapc 1b=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:33.048 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 5.8886855499999999, 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002252291, 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102933 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.049 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.050 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068423+0000", 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068433+0000", 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8959999999999998e-06 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068454+0000", 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0372000000000001e-05 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.051 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068464+0000", 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0828e-05 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068528+0000", 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3744999999999994e-05 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068625+0000", 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6457000000000002e-05 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068735+0000", 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011031600000000001 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.069588+0000", 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085322700000000002 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070578+0000", 2024-09-13T05:03:33.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098980800000000001 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070634+0000", 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6036999999999999e-05 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070678+0000", 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3914e-05 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102997 3.f 3:f4216607:::rbd_header.117c7beb7c1e:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 4.9579287909999996, 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021657629999999998, 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102997 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999180+0000", 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999193+0000", 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2986999999999999e-05 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999209+0000", 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5474000000000001e-05 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999219+0000", 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0745e-05 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999288+0000", 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999449+0000", 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000160435 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999958+0000", 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000509846 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.000669+0000", 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071006499999999996 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001250+0000", 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000581067 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001285+0000", 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4993000000000002e-05 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001348+0000", 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3600000000000001e-05 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.060 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:33.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:33.062 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:33.061+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:33.162 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:33.162+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:33.209 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:33.209 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:33.209 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:33.209 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:33.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:33.263 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:33.263+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:33.271 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:33.325 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:33.325 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:33.325 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:33.326 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:33.326 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:33.335 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:33.364 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:33.364+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:33.395 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:33.395 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:33.412 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:33.412 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:33.428 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:33.428 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:33.444 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:33.445 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.925122583999993, 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.446 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:33.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.448 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:33.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.451 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.452 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.924951278999998, 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:33.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.454 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.455 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:33.456 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.457 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:33.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.998011036000001, 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.459 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.460 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.461 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:33.462 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.463 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.998001256999999, 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.464 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.465 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.466 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.467 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:33.468 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.997987219000002, 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.469 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.470 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.471 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.472 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.997961052999997, 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:33.473 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.474 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.475 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.476 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.893082601000003, 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:33.477 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.478 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.479 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:33.480 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.481 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.890828566000003, 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:33.482 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.483 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.484 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.485 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.890812191999999, 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:33.486 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.487 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.488 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:33.489 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.490 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.888982101000003, 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.491 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.492 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:33.493 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.494 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.495 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.888956976999999, 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.496 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.497 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.498 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.499 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.888946574999999, 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.500 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.501 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:33.502 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:33.503 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.504 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.886628209000001, 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:33.505 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.506 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:33.507 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.508 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.509 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.886603626000003, 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:33.510 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:33.511 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.512 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.513 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.886520662999999, 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.514 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.515 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:33.516 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.517 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:33.518 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.886500081999998, 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:33.519 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.520 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:33.521 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:33.522 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.886483507999998, 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.523 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:33.524 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.525 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.526 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.527 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.886466511000002, 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.528 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.529 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.530 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:33.531 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.885169826000002, 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.532 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.533 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.534 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.535 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.536 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 36.885152941999998, 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:33.537 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.538 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:33.539 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:33.540 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:33.541 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:33.542 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:33.543 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:33.543 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:33.543 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:33.544 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:33.544 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:33.544 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:33.544 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:33.545 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:33.546 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:33.546 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:33.546+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:33.647 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:33.646+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:33.659 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:33.659 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:33.669 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:33.747 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:33.747+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:33.772 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:33.772 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:33.780 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:33.848 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:33.847+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:33.932 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:33.932 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:33.941 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:33.949 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:33.948+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:34.048 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.048 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.051 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:34.050+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:34.057 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:34.152 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:34.150+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:34.159 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.159 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.169 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:34.253 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:34.252+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:34.270 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.270 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.279 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:34.323 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:34.324 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:34.324 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T05:03:34.324 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T05:03:34.354 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:34.354+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:34.381 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.382 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.390 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:34.455 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:34.454+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:34.532 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.532 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.540 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:34.557 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:34.555+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 90, 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "01e1476b-1467-4e0e-b01c-0f8cdf8276fe", 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-13T05:01:45.075461+0000", 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-13T05:03:33.889579+0000", 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-13T05:01:55.160352+0000", 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-13T05:01:47.179750+0000", 2024-09-13T05:03:34.623 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5806080, 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 4, 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T05:03:34.624 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "luminous", 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "reef", 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.817858+0000", 2024-09-13T05:03:34.625 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-13T05:03:34.626 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "17", 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:03:34.627 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 3, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 17, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:03:34.628 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:03:34.629 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.630 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2.630000114440918, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2.630000114440918, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 3, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 3, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 0.79019355773925781, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-13T05:03:34.631 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.981425+0000", 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:03:34.632 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "16", 2024-09-13T05:03:34.633 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:03:34.634 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:03:34.635 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-13T05:03:34.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 6.9000000953674316, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 6.9000000953674316, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.28999999165534973, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 0.79019355773925781, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:02:05.508713+0000", 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:03:34.638 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:03:34.639 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "90", 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 63, 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 90, 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:03:34.640 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:03:34.641 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:03:34.642 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.5399999618530273, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.5399999618530273, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 1.75, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 1.75, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 0.79019355773925781, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-13T05:03:34.643 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "baf16a5c-4204-424b-8d8e-a5b1486aa052", 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.644 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-13T05:03:34.645 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.647 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.648 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.649 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1769628429", 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1769628429", 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1769628429", 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1769628429", 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f3df9a0f-3a9e-4665-9233-718bc0fe1223", 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.650 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.652 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-13T05:03:34.653 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-13T05:03:34.654 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.655 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/827277153", 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/827277153", 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/827277153", 2024-09-13T05:03:34.656 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/827277153", 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1304a876-75c5-454b-a60b-8c9823c4aca8", 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:34.657 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 55, 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.658 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.659 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.661 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/1334123504", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/1334123504", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/1334123504", 2024-09-13T05:03:34.662 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/1334123504", 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "83668012-8ddb-4fe5-bb31-6e78d2723f12", 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:34.663 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6824", 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.664 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6825", 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6826", 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6827", 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6830", 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.666 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6831", 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6828", 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6829", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6825/4134389313", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6827/4134389313", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6831/4134389313", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6829/4134389313", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.668 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f462232f-5144-4652-9139-9039d53e191a", 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:34.669 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6808", 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6809", 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.670 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6810", 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6811", 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.671 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6815", 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6816", 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.672 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6812", 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6813", 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:34.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6809/3681797816", 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6811/3681797816", 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6816/3681797816", 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6813/3681797816", 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1d2d8607-f1d9-45e3-8486-3f5bb23b516b", 2024-09-13T05:03:34.674 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6800", 2024-09-13T05:03:34.675 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6801", 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6802", 2024-09-13T05:03:34.676 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6803", 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.677 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6806", 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6807", 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6804", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6805", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6801/81482652", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6803/81482652", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6807/81482652", 2024-09-13T05:03:34.679 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6805/81482652", 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "7fe0fb7d-0a40-4666-82a1-4e18f53b445c", 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 0, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.680 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6814", 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.681 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6817", 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6818", 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.682 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6819", 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6822", 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.683 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6823", 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6820", 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.684 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6821", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6817/576600432", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6819/576600432", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6823/576600432", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6821/576600432", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.685 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "811503be-abe3-4d64-a094-09ff675f3666", 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 0.3215484619140625, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:03:34.686 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6824", 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6825", 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.687 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6826", 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6827", 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.688 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6830", 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6831", 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.689 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6828", 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6829", 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:34.690 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6825/896779674", 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6827/896779674", 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6831/896779674", 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6829/896779674", 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-13T05:03:34.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358655+0000", 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.692 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.343203+0000", 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.307081+0000", 2024-09-13T05:03:34.693 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.315153+0000", 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.694 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.280902+0000", 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.695 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.302500+0000", 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.354685+0000", 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358223+0000", 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-13T05:03:34.697 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [ 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "pgid": "3.f", 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "mappings": [ 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "from": 7, 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "to": 2 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-13T05:03:34.698 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [ 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:03:34.699 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 20, 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 8 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [ 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:34.700 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 20, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 1 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-13T05:03:34.701 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-13T05:03:34.702 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:34.702 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:34.703 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.703 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.704 INFO:tasks.thrashosds.thrasher:j is {'epoch': 90, 'fsid': '01e1476b-1467-4e0e-b01c-0f8cdf8276fe', 'created': '2024-09-13T05:01:45.075461+0000', 'modified': '2024-09-13T05:03:33.889579+0000', 'last_up_change': '2024-09-13T05:01:55.160352+0000', 'last_in_change': '2024-09-13T05:01:47.179750+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T05:01:55.817858+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.7901935577392578, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T05:01:55.981425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7901935577392578, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T05:02:05.508713+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 63, 'snap_epoch': 90, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.7901935577392578, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}], 'osds': [{'osd': 0, 'uuid': 'baf16a5c-4204-424b-8d8e-a5b1486aa052', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1769628429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1769628429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1769628429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1769628429}]}, 'public_addr': '172.21.15.53:6817/1769628429', 'cluster_addr': '172.21.15.53:6819/1769628429', 'heartbeat_back_addr': '172.21.15.53:6823/1769628429', 'heartbeat_front_addr': '172.21.15.53:6821/1769628429', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f3df9a0f-3a9e-4665-9233-718bc0fe1223', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 827277153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 827277153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 827277153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 827277153}]}, 'public_addr': '172.21.15.53:6801/827277153', 'cluster_addr': '172.21.15.53:6803/827277153', 'heartbeat_back_addr': '172.21.15.53:6807/827277153', 'heartbeat_front_addr': '172.21.15.53:6805/827277153', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1304a876-75c5-454b-a60b-8c9823c4aca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 1334123504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 1334123504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 1334123504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 1334123504}]}, 'public_addr': '172.21.15.53:6809/1334123504', 'cluster_addr': '172.21.15.53:6811/1334123504', 'heartbeat_back_addr': '172.21.15.53:6815/1334123504', 'heartbeat_front_addr': '172.21.15.53:6813/1334123504', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '83668012-8ddb-4fe5-bb31-6e78d2723f12', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6824', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6825', 'nonce': 4134389313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6826', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6827', 'nonce': 4134389313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6830', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6831', 'nonce': 4134389313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6828', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6829', 'nonce': 4134389313}]}, 'public_addr': '172.21.15.53:6825/4134389313', 'cluster_addr': '172.21.15.53:6827/4134389313', 'heartbeat_back_addr': '172.21.15.53:6831/4134389313', 'heartbeat_front_addr': '172.21.15.53:6829/4134389313', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f462232f-5144-4652-9139-9039d53e191a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 3681797816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 3681797816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6815', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6816', 'nonce': 3681797816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 3681797816}]}, 'public_addr': '172.21.15.110:6809/3681797816', 'cluster_addr': '172.21.15.110:6811/3681797816', 'heartbeat_back_addr': '172.21.15.110:6816/3681797816', 'heartbeat_front_addr': '172.21.15.110:6813/3681797816', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1d2d8607-f1d9-45e3-8486-3f5bb23b516b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 81482652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 81482652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 81482652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 81482652}]}, 'public_addr': '172.21.15.110:6801/81482652', 'cluster_addr': '172.21.15.110:6803/81482652', 'heartbeat_back_addr': '172.21.15.110:6807/81482652', 'heartbeat_front_addr': '172.21.15.110:6805/81482652', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7fe0fb7d-0a40-4666-82a1-4e18f53b445c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 576600432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 576600432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 576600432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 576600432}]}, 'public_addr': '172.21.15.110:6817/576600432', 'cluster_addr': '172.21.15.110:6819/576600432', 'heartbeat_back_addr': '172.21.15.110:6823/576600432', 'heartbeat_front_addr': '172.21.15.110:6821/576600432', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '811503be-abe3-4d64-a094-09ff675f3666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3215484619140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6824', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6825', 'nonce': 896779674}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6826', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6827', 'nonce': 896779674}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6830', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6831', 'nonce': 896779674}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6828', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6829', 'nonce': 896779674}]}, 'public_addr': '172.21.15.110:6825/896779674', 'cluster_addr': '172.21.15.110:6827/896779674', 'heartbeat_back_addr': '172.21.15.110:6831/896779674', 'heartbeat_front_addr': '172.21.15.110:6829/896779674', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358655+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.343203+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.307081+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.315153+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.280902+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.302500+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.354685+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.f', 'mappings': [{'from': 7, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 8}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 20, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T05:03:34.705 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.f 2024-09-13T05:03:34.705 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap-items 3.f 2024-09-13T05:03:34.705 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:34.706 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:34.703+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:34.804 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:34.803+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:34.846 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:34.846 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:34.854 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:34.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:03:34.938 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:34.938+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:35.005 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:35.005 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:35.006 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:35.005+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:35.015 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:35.106 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:35.106+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:35.157 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:35.157 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:35.165 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:35.209 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:35.208+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:35.272 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:35.273 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:35.281 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:35.309 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:35.309+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:35.410 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:35.409+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:35.428 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:35.428 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:35.437 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:35.521 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:35.520+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:35.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:35.585 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:35.586 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:35.595 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:35.636 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:35.635+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:35.672 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:35.672 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:35.690 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:35.690 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:35.706 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:35.706 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:35.712 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:35.711+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:35.722 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:35.722 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:35.754 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:35.754 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:35.758 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:35.759 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:35.759 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:35.759 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:35.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:35.770 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:35.771 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:35.787 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:35.787 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:35.807 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:35.807 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:35.813 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:35.812+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:35.880 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:35.880 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:35.880 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:35.889 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:35.903 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:03:35.903 INFO:teuthology.orchestra.run.smithi053.stderr:clear 3.f pg_upmap_items mapping 2024-09-13T05:03:35.913 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:35.913+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:36.014 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:36.013+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:36.026 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:36.026 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663208060000002, 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:03:36.027 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.028 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.029 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.030 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.031 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663185017000004, 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:36.032 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.033 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.034 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.035 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663167684999998, 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.036 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:03:36.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.040 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663151141, 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:36.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.042 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.043 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:03:36.044 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663134016000001, 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.045 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.046 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.047 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.048 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663117788000001, 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.049 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:03:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.050 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.051 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.052 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.053 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.054 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663100976999999, 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.055 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.056 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:03:36.057 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:03:36.058 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:36.059 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663084359000003, 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.060 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.061 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:03:36.062 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.063 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663067781000002, 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.064 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:36.065 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.066 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.067 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.068 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663031427, 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:36.069 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:03:36.070 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.071 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:03:36.072 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.663012483000003, 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.073 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.074 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:03:36.075 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.076 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662995848999998, 2024-09-13T05:03:36.077 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.078 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.079 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.080 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.081 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662969848000003, 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.082 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662937909, 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.087 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662919121999998, 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.091 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:03:36.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.094 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:03:36.095 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662903002, 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.096 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:03:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:03:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:03:36.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662880940999997, 2024-09-13T05:03:36.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:03:36.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:03:36.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.105 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.662863975999997, 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.106 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:03:36.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:03:36.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.350829576999999, 2024-09-13T05:03:36.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:03:36.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:03:36.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.114 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.350810469000002, 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.115 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.116 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.119 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.120 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:36.121 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:36.122 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:36.119+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:36.221 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:36.222+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:36.269 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:36.269 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:36.269 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:36.278 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:36.322 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:36.321+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:36.384 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:36.384 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:36.384 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:36.393 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:36.424 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:36.424+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:36.525 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:36.523+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:36.541 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.541 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 68.736405368000007, 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.542 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.543 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:03:36.544 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.545 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.595336333999995, 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.546 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:03:36.547 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.548 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.549 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.550 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40646 3.4 e37/19 3:23750312:::rbd_data.117c4a06f7bc.00000000000001c6:head v 37'222, mlcod=37'219)", 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 64.362840011000003, 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002132531, 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.551 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177013+0000", 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177012+0000", 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177019+0000", 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9240000000000001e-06 2024-09-13T05:03:36.552 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177033+0000", 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3309000000000001e-05 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177040+0000", 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9460000000000001e-06 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177080+0000", 2024-09-13T05:03:36.553 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0295e-05 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.177100+0000", 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0064999999999998e-05 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179107+0000", 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0020073040000000001 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.554 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:32.179146+0000", 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8590000000000002e-05 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 58.960983540999997, 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.555 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.556 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.557 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.558 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.559 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.181869466999999, 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:36.560 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.561 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.562 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:03:36.563 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.564 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 56.180582465000001, 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.565 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:03:36.566 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.567 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.568 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83276 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'726, mlcod=59'724)", 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 35.56431259, 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039477289999999996, 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.569 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.570 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975540+0000", 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976050+0000", 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050985799999999997 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976069+0000", 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8393000000000002e-05 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.571 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976098+0000", 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9386e-05 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976193+0000", 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4728999999999994e-05 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976217+0000", 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-13T05:03:36.572 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979453+0000", 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032362860000000001 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979488+0000", 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5629000000000001e-05 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.573 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98338 3.f 3:f28d736a:::rbd_data.117cf03415a9.00000000000000b8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.12417121, 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021044290000000001, 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98338 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.574 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415682+0000", 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415680+0000", 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.575 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415690+0000", 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3910000000000004e-06 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415704+0000", 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4653e-05 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415713+0000", 2024-09-13T05:03:36.576 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3960000000000006e-06 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415760+0000", 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7222000000000003e-05 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415850+0000", 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0068999999999994e-05 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.415957+0000", 2024-09-13T05:03:36.577 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0001066 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416597+0000", 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00064032800000000001 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417700+0000", 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001103276 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.578 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417746+0000", 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5439999999999999e-05 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.417787+0000", 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0720999999999999e-05 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98341 3.f 3:f15ef2ff:::rbd_data.117cf03415a9.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:36.579 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 27.123842806999999, 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021618190000000002, 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98341 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.580 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416009+0000", 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.581 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416015+0000", 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8429999999999998e-06 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416028+0000", 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2198000000000001e-05 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.582 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416035+0000", 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1409999999999999e-06 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416207+0000", 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017254999999999999 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416310+0000", 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103286 2024-09-13T05:03:36.583 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416371+0000", 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0238000000000002e-05 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416872+0000", 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050110900000000002 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.584 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418099+0000", 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001227271 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418141+0000", 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2089999999999999e-05 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418172+0000", 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1137999999999999e-05 2024-09-13T05:03:36.585 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98738 3.f 3:f4eda464:::rbd_data.117cf03415a9.000000000000007e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.712088162000001, 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018816306000000001, 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.586 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98738 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.587 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827764+0000", 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827772+0000", 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.585e-06 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827793+0000", 2024-09-13T05:03:36.588 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-05 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827800+0000", 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8619999999999997e-06 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827849+0000", 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9579000000000001e-05 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.589 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827948+0000", 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8382999999999996e-05 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.831301+0000", 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003353698 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846517+0000", 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015215975 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.590 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846526+0000", 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846557+0000", 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1012000000000001e-05 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846581+0000", 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4803999999999998e-05 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.591 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:98738 3.f e66/55 ondisk, result = 0)", 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 26.698836378999999, 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055032170000000004, 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:36.592 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841016+0000", 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.593 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841022+0000", 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.711e-06 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841038+0000", 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6419999999999999e-05 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841050+0000", 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.185e-05 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.594 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846506+0000", 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005456079 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846513+0000", 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846520+0000", 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.595 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.4.0:2844 3.9 e79/19 3:9408067b:::rbd_data.117c394b9965.0000000000000074:1f v 79'410, mlcod=77'407)", 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.556613132000001, 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002066876, 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:36.596 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983240+0000", 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.597 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983249+0000", 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4369999999999996e-06 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983260+0000", 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0465999999999999e-05 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983270+0000", 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0268999999999999e-05 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.598 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983668+0000", 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00039786299999999998 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983694+0000", 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6500999999999999e-05 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985234+0000", 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001539758 2024-09-13T05:03:36.599 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985307+0000", 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3113000000000003e-05 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.4.0:2845 3.9 e79/19 3:9799fd9b:::rbd_data.117c394b9965.000000000000001e:1f v 79'412, mlcod=77'407)", 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.556240193000001, 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002201724, 2024-09-13T05:03:36.600 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.601 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983612+0000", 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983621+0000", 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7639999999999994e-06 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983642+0000", 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1631e-05 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.602 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983649+0000", 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6139999999999997e-06 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.984376+0000", 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072730199999999998 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.984393+0000", 2024-09-13T05:03:36.603 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6738000000000001e-05 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985760+0000", 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013673369999999999 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985815+0000", 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4478000000000003e-05 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.604 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102158 3.f 3:f4db7252:::rbd_data.117c394b9965.0000000000000010:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.482709503000001, 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002237228, 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102158 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.605 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057141+0000", 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057155+0000", 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3583e-05 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057176+0000", 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0815999999999999e-05 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057187+0000", 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0841e-05 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057255+0000", 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8226999999999996e-05 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057343+0000", 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8294999999999994e-05 2024-09-13T05:03:36.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057459+0000", 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115439 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.058518+0000", 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001059286 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059231+0000", 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071358399999999996 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059301+0000", 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9479000000000003e-05 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059381+0000", 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0153000000000003e-05 2024-09-13T05:03:36.610 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1861 3.4 e83/19 3:249fe31d:::rbd_data.117cc3f2292.0000000000000064:1b v 83'980, mlcod=77'977)", 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.5383688539999998, 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022663900000000001, 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:36.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001490+0000", 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7749999999999998e-06 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001497+0000", 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3069999999999997e-06 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001504+0000", 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2819999999999999e-06 2024-09-13T05:03:36.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001592+0000", 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8015000000000003e-05 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001614+0000", 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1163e-05 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.003705+0000", 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002091337 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.003751+0000", 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5890999999999998e-05 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102933 3.f 3:f3a9a226:::rbd_data.117cc3f2292.000000000000000a:head [delete] snapc 1b=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T05:03:36.615 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 9.4714276690000005, 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002252291, 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102933 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068423+0000", 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068433+0000", 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8959999999999998e-06 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068454+0000", 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0372000000000001e-05 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068464+0000", 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0828e-05 2024-09-13T05:03:36.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068528+0000", 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3744999999999994e-05 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068625+0000", 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6457000000000002e-05 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068735+0000", 2024-09-13T05:03:36.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011031600000000001 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.069588+0000", 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085322700000000002 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070578+0000", 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098980800000000001 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070634+0000", 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6036999999999999e-05 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070678+0000", 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3914e-05 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102997 3.f 3:f4216607:::rbd_header.117c7beb7c1e:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T05:03:36.621 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 8.5406709099999993, 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021657629999999998, 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102997 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999180+0000", 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999193+0000", 2024-09-13T05:03:36.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2986999999999999e-05 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999209+0000", 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5474000000000001e-05 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999219+0000", 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0745e-05 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999288+0000", 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999449+0000", 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000160435 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999958+0000", 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000509846 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.000669+0000", 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071006499999999996 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001250+0000", 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000581067 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001285+0000", 2024-09-13T05:03:36.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4993000000000002e-05 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001348+0000", 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3600000000000001e-05 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1954 3.0 e92/19 3:028ef954:::rbd_data.117c4a06f7bc.000000000000001b:12 v 92'547, mlcod=90'542)", 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:35.913328+0000", 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.62652549499999999, 2024-09-13T05:03:36.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021183949999999999, 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.913328+0000", 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.913328+0000", 2024-09-13T05:03:36.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.913327+0000", 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.913336+0000", 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5570000000000001e-06 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.913351+0000", 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4938e-05 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.913359+0000", 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1720000000000007e-06 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.914031+0000", 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00067185400000000005 2024-09-13T05:03:36.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.914053+0000", 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1814000000000002e-05 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.915410+0000", 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001356461 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.915446+0000", 2024-09-13T05:03:36.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6653000000000003e-05 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1956 3.4 e92/19 3:2031bc64:::rbd_data.117c4a06f7bc.0000000000000096:12 v 92'1040, mlcod=91'1037)", 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:35.918410+0000", 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.621443313, 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022804019999999999, 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918410+0000", 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918410+0000", 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918409+0000", 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918416+0000", 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0949999999999998e-06 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918425+0000", 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3650000000000004e-06 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918432+0000", 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3409999999999997e-06 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918517+0000", 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5452999999999997e-05 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918537+0000", 2024-09-13T05:03:36.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9862999999999999e-05 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.920657+0000", 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021194640000000002 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.920690+0000", 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3643000000000003e-05 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.636 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1960 3.4 e92/19 3:234d8752:::rbd_data.117c4a06f7bc.000000000000001a:12 v 92'1048, mlcod=92'1044)", 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:35.928645+0000", 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.61120868299999997, 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024360509999999998, 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928645+0000", 2024-09-13T05:03:36.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928645+0000", 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928643+0000", 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928653+0000", 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928667+0000", 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3645000000000001e-05 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928674+0000", 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1099999999999997e-06 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.929573+0000", 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089939600000000001 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.929596+0000", 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3033000000000001e-05 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.931042+0000", 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014453000000000001 2024-09-13T05:03:36.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.931081+0000", 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9067999999999999e-05 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:36.641 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:36.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:36.643 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:36.642+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:36.743 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:36.742+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:36.790 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:36.790 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:36.790 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:36.790 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:36.798 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:36.844 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:36.844+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:36.945 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:36.944+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:36.954 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:36.954 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:36.954 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:36.954 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:36.954 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:36.964 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:37.046 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:37.046+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:37.078 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:37.078 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:37.078 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:37.078 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:37.078 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.078 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 82.558632356000004, 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:37.080 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.081 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.082 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.083 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:37.084 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.085 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.086 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 82.558461050999995, 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.087 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.088 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.089 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:37.090 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:37.091 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.092 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.093 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.631520807999998, 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.094 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.095 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.096 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.097 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.098 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.631511029000002, 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.099 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.100 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.101 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.102 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:37.103 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.104 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.631496990999999, 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.105 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.106 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:37.107 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.108 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.109 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.631470825000001, 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.110 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.111 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.112 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.113 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.114 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.526592373, 2024-09-13T05:03:37.115 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.116 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.117 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.118 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.119 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:37.120 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.524338338, 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.121 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.122 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.123 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.124 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.125 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.126 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.524321964000002, 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.127 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.128 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:37.129 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.130 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.131 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.522491873, 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:37.132 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.133 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.134 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:37.135 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:37.136 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.136 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.136 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.136 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:37.136 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.137 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.522466749000003, 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.138 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.139 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:37.140 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.141 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.142 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.522456347000002, 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:37.143 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.144 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.145 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:37.146 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.147 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.148 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.520137980999998, 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.149 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.150 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:37.151 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.152 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.153 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.520113397999999, 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.154 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.155 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:37.156 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:37.157 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.158 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.520030435000002, 2024-09-13T05:03:37.159 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:37.160 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.161 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.162 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:37.163 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.164 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.520009854000001, 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:37.165 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.166 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.167 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.168 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:37.169 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.170 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.519993280000001, 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.171 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.172 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:37.173 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:37.174 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.175 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.519976282999998, 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.176 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.177 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:37.178 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.179 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:37.180 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.518679597999999, 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.181 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.182 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.183 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.184 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:37.185 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.186 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 40.518662714000001, 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:37.187 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.188 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:37.189 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.190 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.191 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:37.192 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:37.192 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:37.193 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:37.194 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:37.193+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:37.294 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:37.293+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:37.298 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:37.298 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:37.298 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:37.298 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:37.307 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:37.395 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:37.395+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:37.408 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:37.408 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:37.408 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:37.417 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:37.497 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:37.496+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.561 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317241463000002, 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:37.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:03:37.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317225207999996, 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:03:37.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:03:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:03:37.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317208119, 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:37.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.575 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317191252000001, 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.577 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:03:37.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317141953999993, 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:03:37.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317118684999997, 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.586 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:03:37.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:37.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317102950999995, 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317079970999998, 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:03:37.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:03:37.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.317030904999996, 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:03:37.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:03:37.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:03:37.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316909589999995, 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:03:37.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:03:37.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316854375999995, 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:03:37.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316834087000004, 2024-09-13T05:03:37.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:03:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316816906, 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:03:37.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:03:37.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316729395999999, 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:03:37.622 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:03:37.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316710147999999, 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:03:37.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:03:37.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316649033000004, 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:03:37.631 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:03:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:03:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.635 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316631857999994, 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:03:37.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.638 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:03:37.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316615447999993, 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:03:37.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:03:37.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316594464000005, 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:03:37.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:03:37.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:03:37.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.649 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 70.316577039999999, 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.650 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:37.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:37.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:37.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:37.654 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:37.655 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:37.656 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:37.654+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:37.756 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:37.755+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:37.758 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:37.759 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:37.759 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:37.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:37.819 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:37.857 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:37.856+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:37.870 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:37.870 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:37.870 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:37.871 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:37.871 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:37.880 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:37.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:03:37.969 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:37.970 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:37.970 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:37.968+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:38.001 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:38.001 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:38.018 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:38.018 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:38.035 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:38.036 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:38.053 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:38.054 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:38.059 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:38.059 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:38.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:38.059 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.441063300000003, 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.441044720999997, 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:38.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:38.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.441019926999999, 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:38.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:38.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.441000571000004, 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:38.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.078 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440981884999999, 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:38.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:38.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440956816000003, 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:38.083 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:38.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:38.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.087 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440934441000003, 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.088 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:38.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440913623, 2024-09-13T05:03:38.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:38.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.096 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440888319000003, 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.097 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440869593999999, 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:38.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440851533, 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:38.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440819302000001, 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:38.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:38.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440800142000001, 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:38.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440783039999999, 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:38.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:38.123 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440756743000001, 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:38.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:38.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440730895999998, 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:38.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440714044000003, 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:38.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440697684, 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:38.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.440681560000002, 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:38.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:38.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.145 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 43.44066033, 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:38.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:38.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:38.150 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:38.150 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:38.058+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:38.151 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:38.152 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:38.152 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:38.153 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:38.153 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:38.153 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:38.154 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:38.193 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:38.192+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:38.261 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:38.260+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:38.295 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:38.295 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:38.304 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:38.388 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:38.387+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:38.445 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:38.446 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:38.454 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:38.496 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:38.495+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:38.563 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:38.563+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:38.597 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:38.597 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:38.606 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:38.664 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:38.663+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:38.708 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:38.708 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:38.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:38.765 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:38.764+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:38.818 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:38.819 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:38.827 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:38.869 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:38.868+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:38.925 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:03:38.978 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:38.979 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:38.984 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:38.983+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:38.988 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:39.068 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:39.066+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:39.104 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.104 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.113 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:39.197 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:39.196+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:39.261 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.261 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.269 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:39.270 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:39.268+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:39.370 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:39.370+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:39.416 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.416 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.424 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:39.471 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:39.470+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:39.532 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.532 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.541 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:39.572 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:39.573+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:39.673 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:39.672+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:39.688 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.688 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.697 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:39.774 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:39.773+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:39.804 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.805 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.813 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:39.875 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:39.874+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:39.920 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:39.921 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:39.934 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:39.977 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:39.976+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:40.050 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:40.061 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:40.077 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:40.077+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:40.155 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:40.199 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:40.199+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:40.209 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:40.209 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:40.218 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:40.278 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:40.278+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:40.279 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:40.279 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:40.297 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:40.297 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:40.313 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:40.313 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:40.330 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:40.330 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:40.347 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:40.347 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:40.363 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:40.364 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:40.367 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:40.367 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:40.367 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:40.377 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:40.379 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:40.378+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:40.381 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:40.381 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:40.397 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:40.397 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:40.480 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:40.480+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:40.488 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:40.488 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:40.488 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:40.497 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:40.581 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:40.580+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.085566106000002, 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.605 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.606 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:40.607 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.608 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.609 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:40.610 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 86.085394801000007, 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:40.611 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.612 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:40.613 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.614 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.615 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:40.616 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.158454558000003, 2024-09-13T05:03:40.617 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.618 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:40.619 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.620 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:40.621 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.158444779, 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.622 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.623 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:40.624 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:40.625 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.158430740999997, 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:40.626 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.627 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:40.628 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:40.629 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.630 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.158404574999999, 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.631 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.632 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.633 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.634 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.635 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.053526122999997, 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.636 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:40.637 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.638 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:40.639 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.051272087999997, 2024-09-13T05:03:40.640 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.641 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.642 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:40.643 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.644 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:40.645 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.051255714, 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.646 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.647 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:40.648 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.649 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.049425622999998, 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:40.650 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.651 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:40.652 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.653 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:40.654 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.049400499000001, 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.655 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:40.656 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.657 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.658 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.659 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.049390097, 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.660 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:40.661 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.662 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:40.663 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:40.664 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.047071731000003, 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.665 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.666 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.667 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.668 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.047047147999997, 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.669 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:40.670 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.671 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:40.672 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.673 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.046964185, 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.674 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.675 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.676 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.677 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.046943603999999, 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:40.678 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.679 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.680 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.681 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:40.682 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.046927029999999, 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.683 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.684 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.685 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.686 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.687 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.046910033000003, 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.688 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.689 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:40.690 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.691 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.692 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.045613348000003, 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.693 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.694 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:40.695 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:40.696 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 44.045596463999999, 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:40.697 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:40.698 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:40.699 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:40.700 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:40.701 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:40.703 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:40.703 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:40.702+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:40.803 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:40.802+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:40.814 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:40.814 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:40.814 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:40.823 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:40.904 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:40.904+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:40.932 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:40.932 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:40.932 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-13T05:03:40.932 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-13T05:03:40.932 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-13T05:03:40.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-13T05:03:40.943 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:03:40.971 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:40.972 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:40.972 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:40.972 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:40.972 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:40.980 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:41.019 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:41.019+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:41.063 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:41.063 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-13T05:03:41.063 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31128 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2277376~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842300335999994, 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036301576000000002, 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.086 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31128 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.087 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242390+0000", 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242392+0000", 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2110000000000001e-06 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242398+0000", 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8799999999999996e-06 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242402+0000", 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5990000000000004e-06 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278539+0000", 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036136363999999997 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278609+0000", 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0334999999999997e-05 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278692+0000", 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.2576000000000004e-05 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31130 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2408448~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842284081000003, 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036405631000000001, 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.091 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31130 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242406+0000", 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242408+0000", 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.091e-06 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242414+0000", 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0159999999999997e-06 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242419+0000", 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5499999999999996e-06 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278717+0000", 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036298382999999997 2024-09-13T05:03:41.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278761+0000", 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3686000000000002e-05 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278812+0000", 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1246999999999999e-05 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.095 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31132 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2506752~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842266992000006, 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036581620000000002, 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31132 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242423+0000", 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242425+0000", 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1550000000000001e-06 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242431+0000", 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0569999999999998e-06 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242436+0000", 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8319999999999996e-06 2024-09-13T05:03:41.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278845+0000", 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036408744 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.278916+0000", 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0926999999999994e-05 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279005+0000", 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9278999999999997e-05 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31134 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2605056~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.100 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842250125000007, 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036687610000000002, 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31134 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242440+0000", 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242446+0000", 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.229e-06 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242454+0000", 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6389999999999992e-06 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242458+0000", 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6809999999999997e-06 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279031+0000", 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036572586999999997 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279075+0000", 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4551000000000001e-05 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279128+0000", 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2301e-05 2024-09-13T05:03:41.104 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31136 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2637824~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842200826999999, 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036831944999999998, 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.105 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31136 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242489+0000", 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242492+0000", 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.1e-06 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242502+0000", 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.6150000000000003e-06 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242508+0000", 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4060000000000002e-06 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279162+0000", 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036653552999999998 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279234+0000", 2024-09-13T05:03:41.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1948999999999998e-05 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279321+0000", 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7899000000000007e-05 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31138 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2736128~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842177558000003, 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036926908000000001, 2024-09-13T05:03:41.109 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31138 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242513+0000", 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242512+0000", 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242515+0000", 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2910000000000002e-06 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242519+0000", 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7180000000000004e-06 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242524+0000", 2024-09-13T05:03:41.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279348+0000", 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036824274999999997 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279388+0000", 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0200000000000001e-05 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279440+0000", 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1418999999999997e-05 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31141 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2834432~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842161824000002, 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037096932999999999, 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.114 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31141 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242528+0000", 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242530+0000", 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242543+0000", 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2673e-05 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242547+0000", 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1250000000000003e-06 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279471+0000", 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036924467000000002 2024-09-13T05:03:41.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279544+0000", 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2937999999999999e-05 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279625+0000", 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1003999999999999e-05 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31143 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 2965504~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842138844000004, 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037197657000000002, 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31143 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242551+0000", 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242553+0000", 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.261e-06 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242561+0000", 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1919999999999999e-06 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242593+0000", 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2650000000000001e-05 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279651+0000", 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037058090000000002 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279691+0000", 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9978999999999998e-05 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279749+0000", 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7847999999999999e-05 2024-09-13T05:03:41.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31145 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3063808~8192] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.842089778000002, 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037338608000000002, 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31145 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242601+0000", 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242600+0000", 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242603+0000", 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8909999999999999e-06 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242610+0000", 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7630000000000003e-06 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242615+0000", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3700000000000003e-06 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279783+0000", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037167454000000003 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279853+0000", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0407000000000005e-05 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279939+0000", 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6216000000000003e-05 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31148 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3194880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841968463000001, 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037321249000000001, 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.127 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31148 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242722+0000", 2024-09-13T05:03:41.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242721+0000", 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242730+0000", 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3259999999999996e-06 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242740+0000", 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0062e-05 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242749+0000", 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8559999999999996e-06 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279951+0000", 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037202504999999997 2024-09-13T05:03:41.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.279992+0000", 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1100999999999998e-05 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280043+0000", 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0912999999999999e-05 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31150 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3293184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841913249000001, 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037435847000000001, 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31150 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242777+0000", 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242776+0000", 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242780+0000", 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.3830000000000001e-06 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242787+0000", 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7009999999999996e-06 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242793+0000", 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3059999999999997e-06 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280076+0000", 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037283140999999999 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280147+0000", 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0636999999999995e-05 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280213+0000", 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6314000000000002e-05 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31152 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3391488~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841892959999996, 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037522578000000001, 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.136 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31152 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242797+0000", 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242799+0000", 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3810000000000002e-06 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242806+0000", 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2389999999999999e-06 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242810+0000", 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7910000000000004e-06 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280238+0000", 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037427321999999999 2024-09-13T05:03:41.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280281+0000", 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3034000000000002e-05 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280320+0000", 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9254000000000001e-05 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31154 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3424256~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841875779000006, 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037673159999999997, 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31154 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242815+0000", 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242814+0000", 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242816+0000", 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3089999999999998e-06 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242889+0000", 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2473999999999998e-05 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242896+0000", 2024-09-13T05:03:41.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8090000000000004e-06 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280352+0000", 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037455966 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280422+0000", 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9830000000000001e-05 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280488+0000", 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6166000000000003e-05 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31156 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3522560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841788269000006, 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037690069, 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.145 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31156 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242902+0000", 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242901+0000", 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242905+0000", 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5389999999999999e-06 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242912+0000", 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4100000000000002e-06 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.148 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242917+0000", 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.724e-06 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280514+0000", 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037596451000000003 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280554+0000", 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0575999999999999e-05 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280592+0000", 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31158 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3620864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:41.150 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841769021000005, 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037838863, 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31158 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242921+0000", 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242963+0000", 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.206e-05 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242971+0000", 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.422e-06 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242977+0000", 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.451e-06 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280625+0000", 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037647672 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280695+0000", 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.004e-05 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280760+0000", 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5561000000000005e-05 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31160 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3751936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841707905999996, 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037880440000000001, 2024-09-13T05:03:41.155 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31160 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242982+0000", 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242984+0000", 2024-09-13T05:03:41.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5950000000000001e-06 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242991+0000", 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.331e-06 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242995+0000", 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6870000000000002e-06 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280786+0000", 2024-09-13T05:03:41.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037790329999999997 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280825+0000", 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9079000000000003e-05 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280863+0000", 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7999e-05 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31162 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3850240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841690731, 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038034113000000001, 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31162 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.160 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243000+0000", 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.242999+0000", 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243001+0000", 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.9870000000000002e-06 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243006+0000", 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5390000000000001e-06 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243008+0000", 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.706e-06 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280894+0000", 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037885826999999997 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.280969+0000", 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5147000000000004e-05 2024-09-13T05:03:41.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281034+0000", 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4277999999999997e-05 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31164 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841674320999999, 2024-09-13T05:03:41.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038121503000000001, 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31164 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243016+0000", 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243010+0000", 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999943 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243019+0000", 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0960000000000001e-06 2024-09-13T05:03:41.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243026+0000", 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4429999999999997e-06 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243032+0000", 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3069999999999999e-06 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281060+0000", 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038027361000000003 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281100+0000", 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0082e-05 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281137+0000", 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7740000000000001e-05 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31166 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.168 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841653336999997, 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038268779000000003, 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31166 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243037+0000", 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243039+0000", 2024-09-13T05:03:41.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4090000000000001e-06 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243046+0000", 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8859999999999998e-06 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243050+0000", 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4510000000000002e-06 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281170+0000", 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038119629000000002 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281239+0000", 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9197999999999997e-05 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281306+0000", 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6660000000000002e-05 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:31168 3.9 3:97ac0e28:::rbd_data.117c7aa26b99.0000000000000001:d [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 73.841635913000005, 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038346203000000002, 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 31168 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.173 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243054+0000", 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243056+0000", 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.311e-06 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243064+0000", 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3470000000000001e-06 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.243068+0000", 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7119999999999999e-06 2024-09-13T05:03:41.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281322+0000", 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038254032 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281362+0000", 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9628000000000001e-05 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:27.281401+0000", 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8544000000000002e-05 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:41.179 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:41.179 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:41.178+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:41.279 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:41.279+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:41.281 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:41.281 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:41.281 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:41.281 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:41.289 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:41.380 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:41.378+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:41.391 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:41.391 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:41.391 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:41.400 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:41.483 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:41.482+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925647517000002, 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.544 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.545 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.546 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.547 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.548 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925628938000003, 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:41.549 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:41.550 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.551 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:41.552 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925604143999998, 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.553 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:41.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925584788000002, 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.557 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:41.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.561 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925566101999998, 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:41.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925541033000002, 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.566 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:41.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925518658000001, 2024-09-13T05:03:41.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:41.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925497839999998, 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:41.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:41.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.579 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925472536000001, 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:41.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:41.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925453810999997, 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925435749999998, 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:41.588 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925403519, 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:41.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925384358999999, 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.597 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:41.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.599 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.599 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:41.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:41.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.602 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925367256999998, 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:41.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.92534096, 2024-09-13T05:03:41.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:41.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:41.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925315113000003, 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:41.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:41.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925298261000002, 2024-09-13T05:03:41.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:41.618 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.619 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.620 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925281900999998, 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:41.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925265777, 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.625 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 46.925244546999998, 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:41.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:41.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:41.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:41.634 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:41.635 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:41.636 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:41.635+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:41.736 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:41.734+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:41.777 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:41.778 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:41.778 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:41.778 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:41.787 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:41.836 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:41.836+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:41.929 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:41.929 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:41.929 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:41.930 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:41.930 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:41.934 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:03:41.937 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:41.937+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:41.942 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:42.038 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:42.037+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:42.054 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:42.054 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70864 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.473713429999997, 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038688958000000002, 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.055 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70864 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.056 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579270+0000", 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.491e-06 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579283+0000", 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2653999999999999e-05 2024-09-13T05:03:42.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579289+0000", 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.614870+0000", 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035581387999999999 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615507+0000", 2024-09-13T05:03:42.058 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00063753200000000001 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617957+0000", 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024498380000000002 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70866 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2596864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:42.059 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.473523071000002, 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036734241000000001, 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70866 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:42.060 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579458+0000", 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.061 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579462+0000", 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579471+0000", 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9639999999999992e-06 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579477+0000", 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0000000000000002e-06 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.062 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615818+0000", 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036340823000000001 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616117+0000", 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029901500000000002 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616193+0000", 2024-09-13T05:03:42.063 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5829999999999997e-05 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70869 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.473498833999997, 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036867536999999999, 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.064 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70869 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579482+0000", 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579485+0000", 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3329999999999999e-06 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.066 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579491+0000", 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1580000000000002e-06 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579496+0000", 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8679999999999998e-06 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616223+0000", 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036727187000000001 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.067 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616290+0000", 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7160000000000001e-05 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616350+0000", 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0248999999999999e-05 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70877 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.068 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.472942510999999, 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036468836999999997, 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70877 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.069 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.070 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580043+0000", 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3989999999999997e-06 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580052+0000", 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6319999999999997e-06 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580065+0000", 2024-09-13T05:03:42.071 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3288000000000001e-05 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616372+0000", 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036307147999999997 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616447+0000", 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5296999999999994e-05 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.072 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616508+0000", 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0751999999999997e-05 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70879 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1130496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.472195669000001, 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035893590000000003, 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.073 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70879 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.074 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580785+0000", 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580792+0000", 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9029999999999998e-06 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580804+0000", 2024-09-13T05:03:42.075 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1545999999999999e-05 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580811+0000", 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2609999999999996e-06 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616530+0000", 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035719133 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616604+0000", 2024-09-13T05:03:42.076 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3712000000000002e-05 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616680+0000", 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5957000000000005e-05 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70882 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.472164202999998, 2024-09-13T05:03:42.077 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036032559999999998, 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70882 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.078 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580817+0000", 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580820+0000", 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5720000000000001e-06 2024-09-13T05:03:42.079 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580826+0000", 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4130000000000003e-06 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580831+0000", 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.865e-06 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616703+0000", 2024-09-13T05:03:42.080 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035871694000000003 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616782+0000", 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9369999999999994e-05 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616850+0000", 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8118000000000001e-05 2024-09-13T05:03:42.081 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70891 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1089536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.47214511, 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036169077000000001, 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.082 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70891 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:42.083 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580836+0000", 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580839+0000", 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3760000000000002e-06 2024-09-13T05:03:42.084 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580853+0000", 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4735999999999999e-05 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580859+0000", 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3329999999999996e-06 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.085 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616872+0000", 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036013495999999999 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616945+0000", 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2646999999999997e-05 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617006+0000", 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0853000000000002e-05 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.086 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70893 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.472111284999997, 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036286807999999997, 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70893 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.087 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:42.088 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580873+0000", 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7930000000000002e-06 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580880+0000", 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9970000000000001e-06 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.089 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580885+0000", 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5350000000000004e-06 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036141860999999997 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617091+0000", 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3730000000000001e-05 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.090 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6412e-05 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70895 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1187840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.472000532999999, 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036346900000000001, 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.091 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70895 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.092 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580985+0000", 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4009999999999999e-06 2024-09-13T05:03:42.093 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581000+0000", 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5024999999999999e-05 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581006+0000", 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3080000000000004e-06 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617178+0000", 2024-09-13T05:03:42.094 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036171571999999999 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617328+0000", 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1537000000000002e-05 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.095 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71006 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.467398219000003, 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035821812000000001, 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71006 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.096 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585583+0000", 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585586+0000", 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585593+0000", 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9210000000000003e-06 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585598+0000", 2024-09-13T05:03:42.098 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0159999999999999e-06 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621279+0000", 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035681069000000003 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621346+0000", 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7080000000000001e-05 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.099 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621405+0000", 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9401999999999998e-05 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71008 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.467376958000003, 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035951295000000001, 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.100 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71008 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585604+0000", 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.102 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585607+0000", 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.266e-06 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585613+0000", 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585622+0000", 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7590000000000003e-06 2024-09-13T05:03:42.103 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621431+0000", 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035809331 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621495+0000", 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4611999999999997e-05 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.104 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621556+0000", 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0591999999999999e-05 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71009 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.467355556999998, 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036076001000000003, 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.105 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71009 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.106 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585634+0000", 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.055e-06 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585640+0000", 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4189999999999999e-06 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.107 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585646+0000", 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621580+0000", 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035934440999999998 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.108 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621644+0000", 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3927e-05 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621702+0000", 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7855000000000001e-05 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71011 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.109 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.467326407999998, 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036193110000000001, 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71011 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.110 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.111 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585658+0000", 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7329999999999999e-06 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585666+0000", 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7280000000000001e-06 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585671+0000", 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-06 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.112 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621726+0000", 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036054614999999998 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621790+0000", 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3868000000000006e-05 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621848+0000", 2024-09-13T05:03:42.113 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8913000000000003e-05 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71071 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.466417292000003, 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035743500999999997, 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.114 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71071 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.115 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586566+0000", 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3209999999999999e-06 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586572+0000", 2024-09-13T05:03:42.116 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586578+0000", 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622184+0000", 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035606024999999999 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.117 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622248+0000", 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4691999999999997e-05 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622308+0000", 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9663000000000001e-05 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71096 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:42.118 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.465442504000002, 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035817952, 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71096 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587541+0000", 2024-09-13T05:03:42.120 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587548+0000", 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4679999999999997e-06 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587561+0000", 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2713e-05 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623235+0000", 2024-09-13T05:03:42.121 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035674375000000001 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623300+0000", 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4505999999999995e-05 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623357+0000", 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7661999999999998e-05 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71097 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.465413437999999, 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035944619999999997, 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71097 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.123 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.124 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587575+0000", 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587582+0000", 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1160000000000001e-06 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.125 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587587+0000", 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0490000000000003e-06 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623383+0000", 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035795897 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.126 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623448+0000", 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4604999999999996e-05 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623513+0000", 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5168999999999999e-05 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71115 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.127 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.464585829000001, 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036023959000000001, 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71115 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.128 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588395+0000", 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.129 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588400+0000", 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588410+0000", 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0076e-05 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588417+0000", 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6549999999999998e-06 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.130 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623694+0000", 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035277159000000002 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623760+0000", 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5968000000000003e-05 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624420+0000", 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066017099999999998 2024-09-13T05:03:42.131 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71117 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.464337436000001, 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035938036, 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.132 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71117 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588643+0000", 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588649+0000", 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5210000000000002e-06 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588660+0000", 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1459e-05 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588668+0000", 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5669999999999997e-06 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624447+0000", 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035778841999999998 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624516+0000", 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8774000000000003e-05 2024-09-13T05:03:42.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624582+0000", 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6818999999999998e-05 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71119 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.464018269, 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035764241000000002, 2024-09-13T05:03:42.136 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71119 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588962+0000", 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588968+0000", 2024-09-13T05:03:42.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5770000000000001e-06 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588980+0000", 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1718e-05 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588987+0000", 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.139 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624607+0000", 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035619545000000002 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624670+0000", 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3664999999999994e-05 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624728+0000", 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7433999999999999e-05 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.140 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71120 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1220608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 47.463837534, 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035736393999999998, 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71120 2024-09-13T05:03:42.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589143+0000", 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589149+0000", 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1329999999999998e-06 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589167+0000", 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8236999999999998e-05 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589174+0000", 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4889999999999998e-06 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624752+0000", 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035577723999999998 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624813+0000", 2024-09-13T05:03:42.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1138999999999999e-05 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624881+0000", 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7489000000000004e-05 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.145 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:42.147 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:42.147 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:42.146+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:42.247 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:42.246+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:42.249 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:42.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:42.249 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:42.249 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:42.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:42.348 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:42.347+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:42.399 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:42.399 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:42.400 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:42.400 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:42.400 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:42.408 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:42.409 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:42.453 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:42.453+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:42.543 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:42.544 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:42.550 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:42.550+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:42.553 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:42.553 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:42.553 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:42.553 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:42.553 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.553 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.323026093999999, 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.554 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.555 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.556 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:42.557 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.558 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.322097984999999, 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.559 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.560 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:42.561 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:42.562 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.563 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.320999576000006, 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.564 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:42.565 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.566 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:42.567 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.568 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.314939615, 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.569 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:42.570 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.571 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:42.572 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:42.573 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.574 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.314930619999998, 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.575 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.576 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.577 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.578 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.314922249000006, 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.579 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.580 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.581 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:42.582 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.583 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.314912227999997, 2024-09-13T05:03:42.584 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.585 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.586 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.587 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.587 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.587 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:42.587 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:42.587 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.588 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.589 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.314896504999993, 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.590 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.591 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:42.592 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:42.593 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.594 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.312994336000003, 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.595 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.596 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:42.597 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.598 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.312986377000001, 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.599 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:42.600 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.601 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.602 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.603 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.312978689000005, 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.604 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.605 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.606 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.607 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.312967997000001, 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.608 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:42.609 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.612 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.312959755999998, 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:42.613 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.614 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.615 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:42.616 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.617 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.312936094999998, 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.620 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.621 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.622 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.623 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.624 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.311293636000002, 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.625 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.626 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:42.627 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:42.628 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.311285402999999, 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.629 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.630 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:42.631 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.632 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.633 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.311062673999999, 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.634 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.635 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.636 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:42.637 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.311050124999994, 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:42.638 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.639 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:42.640 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:42.641 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.642 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.309010899, 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.643 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.644 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.645 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:42.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 78.309001120000005, 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.649 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:42.650 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:42.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:42.652 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:42.654 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:42.655 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:42.656 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:42.656 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:42.657 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:42.657 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:42.658 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:42.658 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:42.658 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:42.658 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:42.659 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:42.659 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:42.659 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:42.660 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:42.660 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:42.655+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:42.660 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:42.660 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:42.757 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:42.756+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:42.767 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:42.776 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:42.861 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:42.861+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:42.925 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:42.933 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:42.975 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:42.974+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:43.060 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:43.059+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:43.088 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.089 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.097 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:43.161 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:43.160+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:43.204 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.205 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.213 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:43.262 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:43.261+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:43.320 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.321 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.329 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:43.363 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:43.362+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:43.436 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.486 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.494 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:43.494 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:43.492+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:43.594 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:43.593+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:43.602 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.602 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.610 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:43.694 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:43.693+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:43.757 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.757 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.765 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:43.809 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:43.809+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:43.872 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:43.872 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:43.882 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:43.895 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:43.894+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:43.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-13T05:03:44.009 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:44.009+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:44.045 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.045 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.057 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:44.097 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:44.096+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:44.197 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:44.196+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:44.205 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.205 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.214 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:44.297 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:44.297+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:44.360 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.360 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.369 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:44.399 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:44.398+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:44.472 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.472 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.480 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:44.523 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:44.522+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:44.582 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.582 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.590 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:44.632 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:44.631+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:44.670 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:44.713 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:44.713+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:44.733 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.733 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.741 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:44.795 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:44.796 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:44.803 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:44.802+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:44.812 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:44.813 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:44.828 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:44.828 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:44.844 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:44.844 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:44.844 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:44.845 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:44.853 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:44.860 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:44.860 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:44.876 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:44.877 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:44.882 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is set 2024-09-13T05:03:44.904 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:44.904+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:44.908 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:44.908 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:44.936 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:44.937 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:44.975 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:44.975 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:44.976 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:44.976 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:44.976 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:44.985 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:45.005 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:45.005+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:45.107 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:45.106+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70965 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2818048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511669695999998, 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041397116999999997, 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.130 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70965 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616950+0000", 2024-09-13T05:03:45.131 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616949+0000", 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616952+0000", 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4339999999999999e-06 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.132 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616959+0000", 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5390000000000002e-06 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616964+0000", 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7310000000000001e-06 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658206+0000", 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041241762000000001 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.133 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658278+0000", 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1693999999999995e-05 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658347+0000", 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9420999999999998e-05 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.134 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70968 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2727936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511651117, 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041563513000000003, 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70968 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616968+0000", 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616975+0000", 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3520000000000001e-06 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616983+0000", 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9389999999999997e-06 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616989+0000", 2024-09-13T05:03:45.137 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2789999999999998e-06 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658383+0000", 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041394168000000002 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658460+0000", 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7506e-05 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.138 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658532+0000", 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1567000000000001e-05 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70970 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2777088~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511626323000002, 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041652568000000001, 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.139 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70970 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.140 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616993+0000", 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616995+0000", 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2230000000000001e-06 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617002+0000", 2024-09-13T05:03:45.141 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4150000000000002e-06 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617007+0000", 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5889999999999996e-06 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658553+0000", 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041545795000000003 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.142 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658590+0000", 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6878999999999997e-05 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658646+0000", 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6063999999999997e-05 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70971 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2826240~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511606966999999, 2024-09-13T05:03:45.143 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041762123999999998, 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70971 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.144 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617013+0000", 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617012+0000", 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617014+0000", 2024-09-13T05:03:45.145 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.339e-06 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617022+0000", 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.554e-06 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8590000000000004e-06 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.146 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658672+0000", 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041645329000000002 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658731+0000", 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8594000000000002e-05 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658775+0000", 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3888999999999999e-05 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.147 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70986 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2686976~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511588281000002, 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041937452, 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.148 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70986 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.149 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617031+0000", 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617033+0000", 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1859999999999999e-06 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617041+0000", 2024-09-13T05:03:45.150 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.405e-06 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617052+0000", 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0335e-05 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658812+0000", 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041760261 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.151 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658888+0000", 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6281999999999999e-05 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658969+0000", 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0384999999999997e-05 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70988 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2736128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:45.152 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511563211999999, 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042035298999999998, 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70988 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:45.153 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617056+0000", 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617058+0000", 2024-09-13T05:03:45.154 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4210000000000002e-06 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617065+0000", 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6819999999999997e-06 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617074+0000", 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2199999999999998e-06 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.155 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.658997+0000", 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.041922398999999999 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659046+0000", 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9515000000000003e-05 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659092+0000", 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5516999999999998e-05 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.156 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70989 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511540836999998, 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042204097000000003, 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70989 2024-09-13T05:03:45.157 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617079+0000", 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.158 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617078+0000", 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617080+0000", 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1239999999999999e-06 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.159 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617087+0000", 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2689999999999996e-06 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617095+0000", 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0930000000000003e-06 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659132+0000", 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042037499999999998 2024-09-13T05:03:45.160 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659211+0000", 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8720999999999994e-05 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659283+0000", 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1771000000000001e-05 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.161 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70993 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511520019000002, 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042301204000000002, 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.162 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70993 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617100+0000", 2024-09-13T05:03:45.163 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617099+0000", 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617101+0000", 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.17e-06 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.164 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617108+0000", 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7329999999999997e-06 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617113+0000", 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3399999999999997e-06 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659314+0000", 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042200564000000003 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.165 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659358+0000", 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3655999999999997e-05 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659401+0000", 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3111000000000001e-05 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70994 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.166 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511494714999998, 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042452901000000001, 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70994 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.167 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617125+0000", 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617124+0000", 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.168 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617127+0000", 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5689999999999999e-06 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617134+0000", 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3900000000000004e-06 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617139+0000", 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0050000000000004e-06 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.169 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659435+0000", 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042295280999999997 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659508+0000", 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2927000000000002e-05 2024-09-13T05:03:45.170 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659578+0000", 2024-09-13T05:03:45.171 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.0171000000000003e-05 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70995 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:45.172 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511475990000001, 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042609474000000001, 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70995 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.173 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617144+0000", 2024-09-13T05:03:45.174 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617143+0000", 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.175 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617145+0000", 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1799999999999999e-06 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617153+0000", 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3289999999999996e-06 2024-09-13T05:03:45.176 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7670000000000003e-06 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659611+0000", 2024-09-13T05:03:45.177 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042453139000000001 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659684+0000", 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3418999999999996e-05 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659753+0000", 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9043999999999998e-05 2024-09-13T05:03:45.178 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71010 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2703360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511457929000002, 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042764669999999998, 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.179 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71010 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617162+0000", 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.180 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617161+0000", 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617176+0000", 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4657999999999999e-05 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617184+0000", 2024-09-13T05:03:45.181 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.191e-06 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617189+0000", 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9649999999999999e-06 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659780+0000", 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042591072000000001 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.182 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659859+0000", 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9132000000000001e-05 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659926+0000", 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7031000000000002e-05 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71012 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2752512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:45.183 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511425697999996, 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042895721999999997, 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71012 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.184 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617194+0000", 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617193+0000", 2024-09-13T05:03:45.185 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617196+0000", 2024-09-13T05:03:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2069999999999998e-06 2024-09-13T05:03:45.187 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617204+0000", 2024-09-13T05:03:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.0330000000000008e-06 2024-09-13T05:03:45.188 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617209+0000", 2024-09-13T05:03:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0869999999999997e-06 2024-09-13T05:03:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.189 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.659960+0000", 2024-09-13T05:03:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042751141999999999 2024-09-13T05:03:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.190 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660028+0000", 2024-09-13T05:03:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8503000000000005e-05 2024-09-13T05:03:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.191 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660090+0000", 2024-09-13T05:03:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1205999999999996e-05 2024-09-13T05:03:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.192 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71013 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2801664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511406538000003, 2024-09-13T05:03:45.193 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043074860999999999, 2024-09-13T05:03:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.194 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71013 2024-09-13T05:03:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.195 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:45.196 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617213+0000", 2024-09-13T05:03:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.197 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.198 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.198 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617215+0000", 2024-09-13T05:03:45.198 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:45.198 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.198 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.199 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.199 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617221+0000", 2024-09-13T05:03:45.199 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.2430000000000002e-06 2024-09-13T05:03:45.199 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.199 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.200 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617226+0000", 2024-09-13T05:03:45.200 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7609999999999998e-06 2024-09-13T05:03:45.200 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.200 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.200 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660140+0000", 2024-09-13T05:03:45.201 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.042914458000000003 2024-09-13T05:03:45.201 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.201 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.201 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.201 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660216+0000", 2024-09-13T05:03:45.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5998e-05 2024-09-13T05:03:45.202 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.202 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.202 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.202 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660288+0000", 2024-09-13T05:03:45.202 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1581999999999993e-05 2024-09-13T05:03:45.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.203 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.203 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.203 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.203 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.204 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71014 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2711552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.204 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:45.204 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511389436000002, 2024-09-13T05:03:45.204 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043191628000000003, 2024-09-13T05:03:45.204 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.204 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.205 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.205 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.205 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.205 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71014 2024-09-13T05:03:45.205 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.205 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.206 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.206 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.206 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:45.206 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.206 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617230+0000", 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.207 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617239+0000", 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.3940000000000002e-06 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.7710000000000006e-06 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.208 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617252+0000", 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9690000000000002e-06 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660316+0000", 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043064384999999997 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660361+0000", 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4743000000000002e-05 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.209 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660422+0000", 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0748999999999997e-05 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71016 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2760704~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511363138999997, 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043316195000000002, 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.210 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71016 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.211 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617256+0000", 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617258+0000", 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.212 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617267+0000", 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.5490000000000006e-06 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617278+0000", 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0927000000000001e-05 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660452+0000", 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043174838 2024-09-13T05:03:45.213 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660517+0000", 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4926999999999997e-05 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660573+0000", 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5260999999999998e-05 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.214 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71018 3.c 3:3244bd76:::rbd_data.117c259cf1fe.000000000000000b:1e [read 2809856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511337292, 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043415088999999997, 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.215 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71018 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.216 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617282+0000", 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617284+0000", 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.384e-06 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617290+0000", 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1460000000000001e-06 2024-09-13T05:03:45.217 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617295+0000", 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.476e-06 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660601+0000", 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043306070000000002 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.218 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660656+0000", 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4623999999999997e-05 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660697+0000", 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.1828e-05 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71027 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 186368~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:45.219 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511320439999999, 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043467157999999999, 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71027 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:45.220 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617299+0000", 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617301+0000", 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0660000000000002e-06 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617307+0000", 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9340000000000003e-06 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617311+0000", 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4619999999999998e-06 2024-09-13T05:03:45.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660616+0000", 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043305038999999997 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660695+0000", 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8448000000000005e-05 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660766+0000", 2024-09-13T05:03:45.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1551000000000007e-05 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71028 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 8192~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511304080000002, 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043640709999999999, 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.224 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71028 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617315+0000", 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617317+0000", 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.041e-06 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.226 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617323+0000", 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6030000000000004e-06 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617327+0000", 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6609999999999999e-06 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660790+0000", 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043462818 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660859+0000", 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8862999999999994e-05 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660956+0000", 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7084999999999996e-05 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71041 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 106496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.228 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511287955999997, 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829106999999999, 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71041 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:45.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617332+0000", 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617331+0000", 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617333+0000", 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2390000000000001e-06 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617344+0000", 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0251000000000001e-05 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617348+0000", 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8040000000000001e-06 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.660997+0000", 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.04364809 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661085+0000", 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.8646999999999993e-05 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661161+0000", 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5520999999999997e-05 2024-09-13T05:03:45.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71047 3.1c 3:3c2a9ef6:::rbd_data.117cb402b822.0000000000000000:1d [read 67584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 50.511266726000002, 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043985191999999999, 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.233 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71047 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617353+0000", 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617352+0000", 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617355+0000", 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617360+0000", 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8660000000000002e-06 2024-09-13T05:03:45.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617365+0000", 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6290000000000001e-06 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661194+0000", 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.043829211999999999 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661269+0000", 2024-09-13T05:03:45.236 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4958999999999998e-05 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.661338+0000", 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8807999999999996e-05 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.237 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:45.239 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:45.239 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:45.238+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:45.339 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:45.339+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:45.380 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:45.381 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:45.381 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:45.381 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:45.390 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:45.441 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:45.441+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:45.531 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:45.532 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:45.532 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:45.532 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:45.532 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:45.540 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:45.542 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:45.541+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:45.643 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:45.642+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:45.644 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:45.644 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:45.644 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:45.644 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:45.644 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70864 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2547712~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.063654595999999, 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038688958000000002, 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70864 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.645 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.646 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579268+0000", 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.647 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579270+0000", 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.491e-06 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579283+0000", 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2653999999999999e-05 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579289+0000", 2024-09-13T05:03:45.648 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4999999999999999e-06 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.614870+0000", 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035581387999999999 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615507+0000", 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00063753200000000001 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.649 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.650 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.650 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617957+0000", 2024-09-13T05:03:45.650 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0024498380000000002 2024-09-13T05:03:45.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.650 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.650 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.651 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.651 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.651 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70866 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2596864~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.651 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:45.651 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.063464236999998, 2024-09-13T05:03:45.651 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036734241000000001, 2024-09-13T05:03:45.652 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.652 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.652 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.652 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.652 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.652 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70866 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.653 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579459+0000", 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.654 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579458+0000", 2024-09-13T05:03:45.655 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:45.655 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.655 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.655 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.655 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579462+0000", 2024-09-13T05:03:45.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3379999999999998e-06 2024-09-13T05:03:45.656 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.656 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.656 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.656 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579471+0000", 2024-09-13T05:03:45.656 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.9639999999999992e-06 2024-09-13T05:03:45.657 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.657 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.657 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.657 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579477+0000", 2024-09-13T05:03:45.657 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0000000000000002e-06 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.615818+0000", 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036340823000000001 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.658 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616117+0000", 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00029901500000000002 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616193+0000", 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5829999999999997e-05 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.659 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70869 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2646016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.06344, 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036867536999999999, 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70869 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.660 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579483+0000", 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.661 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579482+0000", 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579485+0000", 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3329999999999999e-06 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579491+0000", 2024-09-13T05:03:45.662 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1580000000000002e-06 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.579496+0000", 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8679999999999998e-06 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616223+0000", 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036727187000000001 2024-09-13T05:03:45.663 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616290+0000", 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7160000000000001e-05 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616350+0000", 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0248999999999999e-05 2024-09-13T05:03:45.664 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70877 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1081344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.062883677000002, 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036468836999999997, 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.665 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70877 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.666 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:45.667 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.668 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.668 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.668 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.668 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580039+0000", 2024-09-13T05:03:45.668 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580043+0000", 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3989999999999997e-06 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.669 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.670 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.670 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580052+0000", 2024-09-13T05:03:45.670 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.6319999999999997e-06 2024-09-13T05:03:45.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.670 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580065+0000", 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3288000000000001e-05 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.671 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616372+0000", 2024-09-13T05:03:45.672 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036307147999999997 2024-09-13T05:03:45.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.672 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.672 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.672 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616447+0000", 2024-09-13T05:03:45.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5296999999999994e-05 2024-09-13T05:03:45.673 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.673 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.673 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.673 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616508+0000", 2024-09-13T05:03:45.673 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0751999999999997e-05 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70879 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1130496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.062136834999997, 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035893590000000003, 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.674 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70879 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.675 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580786+0000", 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580785+0000", 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580792+0000", 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9029999999999998e-06 2024-09-13T05:03:45.676 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580804+0000", 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1545999999999999e-05 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580811+0000", 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2609999999999996e-06 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.677 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616530+0000", 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035719133 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616604+0000", 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3712000000000002e-05 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616680+0000", 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5957000000000005e-05 2024-09-13T05:03:45.678 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70882 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1179648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.062105369000001, 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036032559999999998, 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.679 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70882 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580818+0000", 2024-09-13T05:03:45.680 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580817+0000", 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580820+0000", 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5720000000000001e-06 2024-09-13T05:03:45.681 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580826+0000", 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4130000000000003e-06 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580831+0000", 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.865e-06 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616703+0000", 2024-09-13T05:03:45.682 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035871694000000003 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616782+0000", 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.9369999999999994e-05 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616850+0000", 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8118000000000001e-05 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.683 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70891 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1089536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.062086276000002, 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036169077000000001, 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70891 2024-09-13T05:03:45.684 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580837+0000", 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.685 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580836+0000", 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580839+0000", 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3760000000000002e-06 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580853+0000", 2024-09-13T05:03:45.686 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.4735999999999999e-05 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580859+0000", 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.3329999999999996e-06 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616872+0000", 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036013495999999999 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.687 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.616945+0000", 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2646999999999997e-05 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617006+0000", 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0853000000000002e-05 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.688 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70893 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1138688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.062052451, 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036286807999999997, 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.689 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70893 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.691 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.691 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.691 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580870+0000", 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580873+0000", 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7930000000000002e-06 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.693 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580880+0000", 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9970000000000001e-06 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580885+0000", 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5350000000000004e-06 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617027+0000", 2024-09-13T05:03:45.694 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036141860999999997 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617091+0000", 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3730000000000001e-05 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.695 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617157+0000", 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6412e-05 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:70895 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1187840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.061941699000002, 2024-09-13T05:03:45.696 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036346900000000001, 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 70895 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.697 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.698 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580981+0000", 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.580985+0000", 2024-09-13T05:03:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4009999999999999e-06 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581000+0000", 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5024999999999999e-05 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.581006+0000", 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3080000000000004e-06 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.700 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617178+0000", 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036171571999999999 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617247+0000", 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.701 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.617328+0000", 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1537000000000002e-05 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71006 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2785280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.057339384999999, 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035821812000000001, 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.702 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71006 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585584+0000", 2024-09-13T05:03:45.703 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585583+0000", 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585586+0000", 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7389999999999999e-06 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.704 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585593+0000", 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.9210000000000003e-06 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585598+0000", 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0159999999999999e-06 2024-09-13T05:03:45.705 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621279+0000", 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035681069000000003 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621346+0000", 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7080000000000001e-05 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.706 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621405+0000", 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9401999999999998e-05 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71008 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2695168~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.057318123999998, 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035951295000000001, 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.707 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71008 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.708 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585605+0000", 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585604+0000", 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585607+0000", 2024-09-13T05:03:45.709 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.266e-06 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585613+0000", 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1800000000000001e-06 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.710 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585622+0000", 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7590000000000003e-06 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621431+0000", 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035809331 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621495+0000", 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4611999999999997e-05 2024-09-13T05:03:45.711 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621556+0000", 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.0591999999999999e-05 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71009 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2744320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.057296723, 2024-09-13T05:03:45.712 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036076001000000003, 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71009 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:45.713 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585626+0000", 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.714 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585634+0000", 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.055e-06 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585640+0000", 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4189999999999999e-06 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585646+0000", 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6760000000000004e-06 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.715 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621580+0000", 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035934440999999998 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621644+0000", 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3927e-05 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621702+0000", 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7855000000000001e-05 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.716 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71011 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.057267574000001, 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036193110000000001, 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.717 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71011 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.718 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585655+0000", 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585658+0000", 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7329999999999999e-06 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585666+0000", 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.7280000000000001e-06 2024-09-13T05:03:45.719 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.585671+0000", 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7840000000000003e-06 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621726+0000", 2024-09-13T05:03:45.720 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036054614999999998 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621790+0000", 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3868000000000006e-05 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.621848+0000", 2024-09-13T05:03:45.721 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.8913000000000003e-05 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71071 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1327104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.056358457999998, 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035743500999999997, 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.722 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71071 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.723 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586564+0000", 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586566+0000", 2024-09-13T05:03:45.724 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3209999999999999e-06 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586572+0000", 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7389999999999996e-06 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.586578+0000", 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.4269999999999999e-06 2024-09-13T05:03:45.725 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622184+0000", 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035606024999999999 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622248+0000", 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4691999999999997e-05 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.622308+0000", 2024-09-13T05:03:45.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.9663000000000001e-05 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71096 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1204224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.055383669999998, 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035817952, 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.727 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71096 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587539+0000", 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587541+0000", 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7870000000000002e-06 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587548+0000", 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4679999999999997e-06 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587561+0000", 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2713e-05 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623235+0000", 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035674375000000001 2024-09-13T05:03:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623300+0000", 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4505999999999995e-05 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623357+0000", 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7661999999999998e-05 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71097 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1253376~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.055354604000001, 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035944619999999997, 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71097 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.733 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587568+0000", 2024-09-13T05:03:45.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587575+0000", 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.3130000000000001e-06 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587582+0000", 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1160000000000001e-06 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.587587+0000", 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0490000000000003e-06 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623383+0000", 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035795897 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623448+0000", 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.4604999999999996e-05 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623513+0000", 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5168999999999999e-05 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.737 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71115 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1212416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.054526995000003, 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036023959000000001, 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71115 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588396+0000", 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588395+0000", 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:45.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588400+0000", 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8539999999999996e-06 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588410+0000", 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0076e-05 2024-09-13T05:03:45.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588417+0000", 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6549999999999998e-06 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623694+0000", 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035277159000000002 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.623760+0000", 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.5968000000000003e-05 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624420+0000", 2024-09-13T05:03:45.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00066017099999999998 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71117 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1261568~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.054278601999997, 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035938036, 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.743 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71117 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588644+0000", 2024-09-13T05:03:45.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588643+0000", 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588649+0000", 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5210000000000002e-06 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588660+0000", 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1459e-05 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588668+0000", 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.5669999999999997e-06 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624447+0000", 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035778841999999998 2024-09-13T05:03:45.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624516+0000", 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.8774000000000003e-05 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624582+0000", 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6818999999999998e-05 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71119 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1310720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.053959435000003, 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035764241000000002, 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71119 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588963+0000", 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588962+0000", 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:45.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588968+0000", 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.5770000000000001e-06 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588980+0000", 2024-09-13T05:03:45.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1718e-05 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.588987+0000", 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.2880000000000004e-06 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624607+0000", 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035619545000000002 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624670+0000", 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.3664999999999994e-05 2024-09-13T05:03:45.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624728+0000", 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7433999999999999e-05 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.753 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:71120 3.a 3:5032a2e6:::rbd_data.117c259cf1fe.000000000000000c:1e [read 1220608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54)", 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 51.053778700000002, 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035736393999999998, 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:45.754 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 71120 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:45.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589144+0000", 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:45.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589143+0000", 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999999 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589149+0000", 2024-09-13T05:03:45.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1329999999999998e-06 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589167+0000", 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.8236999999999998e-05 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.589174+0000", 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.4889999999999998e-06 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624752+0000", 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.035577723999999998 2024-09-13T05:03:45.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624813+0000", 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.1138999999999999e-05 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:45.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:54.624881+0000", 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.7489000000000004e-05 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:45.761 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:45.763 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:45.764 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:45.762+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:45.863 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:45.863+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:45.907 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:45.907 INFO:teuthology.orchestra.run.smithi110.stdout: "num_ops": 0 2024-09-13T05:03:45.907 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:45.915 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:45.964 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:45.964+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:46.065 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:46.065 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:46.065 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:46.065 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:46.065 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:46.065 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:46.064+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:46.073 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:46.073 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:46.073 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-13T05:03:46.073 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-13T05:03:46.074 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:46.166 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:46.166+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.988645962000007, 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:46.219 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.220 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.221 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.222 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:46.223 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:46.224 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.224 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.224 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.224 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:46.224 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:46.224 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.225 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.225 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.225 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.225 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.225 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.225 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:46.226 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.987717853000007, 2024-09-13T05:03:46.226 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:46.226 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.226 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.226 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.227 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:46.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.228 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.228 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.228 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.228 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:46.228 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.229 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.230 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:46.230 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:46.230 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.230 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.230 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:46.231 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:46.231 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.231 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.231 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.231 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:46.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.233 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.233 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.233 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:46.233 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:46.233 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.234 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.235 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.235 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.235 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.235 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:46.235 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.986619443999999, 2024-09-13T05:03:46.235 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:46.236 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.236 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.236 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.236 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.236 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.236 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:46.237 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.237 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.237 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.237 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.237 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:46.237 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.238 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.238 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.238 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:46.238 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.238 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.239 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.239 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:46.239 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.239 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.239 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.240 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:46.240 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:46.240 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.240 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.240 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:46.241 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:46.241 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.241 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.241 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.241 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:46.242 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.243 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.243 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.243 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:46.243 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:46.243 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.244 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.244 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.244 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:46.244 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:46.244 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.244 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.245 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.245 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.245 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.245 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.245 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:46.245 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.980559482999993, 2024-09-13T05:03:46.246 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:46.246 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.246 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:46.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.246 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.247 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.247 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:46.247 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.247 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.247 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.247 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.248 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:46.248 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.248 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.248 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.249 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:46.249 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.249 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.249 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:46.250 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.251 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.252 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:46.253 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.254 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.255 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.980550488000006, 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.256 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:46.258 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.259 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.980542116999999, 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.261 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:46.262 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.263 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.264 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.265 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.980532096000005, 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.266 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.267 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:46.268 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:46.269 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.980516373, 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.270 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.271 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:46.272 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.273 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:46.274 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.978614203999996, 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.275 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:46.276 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.277 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.278 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.978606244999995, 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.279 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:46.280 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.281 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.282 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.978598556999998, 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.283 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:46.284 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.285 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.286 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.287 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.978587864999994, 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:46.288 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.289 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.290 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.291 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.978579624000005, 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.292 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.293 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.294 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:46.295 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.296 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.978555963000005, 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.297 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.298 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:46.299 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.300 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.976913503999995, 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.301 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.302 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:46.303 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.304 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.305 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.976905271000007, 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:46.306 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.307 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.308 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.309 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.976682542000006, 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.310 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.311 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.312 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.313 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.314 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.976669993000002, 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.315 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.316 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.317 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.318 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.974630766999994, 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.319 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.320 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.321 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:46.322 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:46.323 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.324 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 81.974620987999998, 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.325 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.326 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.327 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:46.328 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:46.329 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:46.330 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:46.332 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:46.332 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:46.331+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:03:46.363 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "epoch": 102, 2024-09-13T05:03:46.363 INFO:teuthology.orchestra.run.smithi053.stdout: "fsid": "01e1476b-1467-4e0e-b01c-0f8cdf8276fe", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "created": "2024-09-13T05:01:45.075461+0000", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "modified": "2024-09-13T05:03:45.946930+0000", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_up_change": "2024-09-13T05:01:55.160352+0000", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "last_in_change": "2024-09-13T05:01:47.179750+0000", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_num": 5806080, 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_set": [ 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "nodeep-scrub", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "noscrub", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "pglog_hardlimit", 2024-09-13T05:03:46.364 INFO:teuthology.orchestra.run.smithi053.stdout: "purged_snapdirs", 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "recovery_deletes", 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "sortbitwise" 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_version": 4, 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "full_ratio": 0.94999998807907104, 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_snapshot": "", 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_max": 3, 2024-09-13T05:03:46.365 INFO:teuthology.orchestra.run.smithi053.stdout: "max_osd": 8, 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "require_min_compat_client": "luminous", 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "min_compat_client": "jewel", 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "require_osd_release": "reef", 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "allow_crimson": false, 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pools": [ 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 1, 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "rbd", 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.817858+0000", 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:03:46.366 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 8, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 8, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 8, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 8, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 8, 2024-09-13T05:03:46.367 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "17", 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:03:46.368 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 3, 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 17, 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:03:46.369 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:03:46.370 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:03:46.371 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:03:46.372 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 2.630000114440918, 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 2.630000114440918, 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 3, 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 3, 2024-09-13T05:03:46.373 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 0.79019355773925781, 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 2, 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": ".mgr", 2024-09-13T05:03:46.374 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:01:55.981425+0000", 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 1, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool", 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:03:46.375 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:03:46.376 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:03:46.376 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:03:46.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:03:46.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 1, 2024-09-13T05:03:46.376 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 1, 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 1, 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 1, 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 1, 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:03:46.377 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "16", 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:03:46.378 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 0, 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 0, 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:03:46.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:03:46.380 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:03:46.381 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:03:46.382 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:03:46.383 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:03:46.383 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:03:46.383 INFO:teuthology.orchestra.run.smithi053.stdout: "options": { 2024-09-13T05:03:46.383 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_max": 32, 2024-09-13T05:03:46.383 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_min": 1 2024-09-13T05:03:46.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "mgr": {} 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 6.9000000953674316, 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 6.9000000953674316, 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 0.28999999165534973, 2024-09-13T05:03:46.384 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 0.79019355773925781, 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_name": "pool_client.0", 2024-09-13T05:03:46.385 INFO:teuthology.orchestra.run.smithi053.stdout: "create_time": "2024-09-13T05:02:05.508713+0000", 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flags": 8193, 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "type": 1, 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 2, 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "min_size": 1, 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_rule": 0, 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_count": 0, 2024-09-13T05:03:46.386 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_target": 0, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_barrier": 0, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "object_hash": 2, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_autoscale_mode": "off", 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num": 32, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num": 32, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_placement_num_target": 32, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_target": 32, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_num_pending": 32, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "last_pg_merge_meta": { 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "source_pgid": "0.0", 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "ready_epoch": 0, 2024-09-13T05:03:46.387 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_started": 0, 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_epoch_clean": 0, 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "source_version": "0'0", 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "target_version": "0'0" 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_change": "102", 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend": "0", 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "last_force_op_resend_preluminous": "0", 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "auid": 0, 2024-09-13T05:03:46.388 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_mode": "selfmanaged", 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_seq": 75, 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "snap_epoch": 102, 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "pool_snaps": [], 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps": "[]", 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_bytes": 0, 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "quota_max_objects": 0, 2024-09-13T05:03:46.389 INFO:teuthology.orchestra.run.smithi053.stdout: "tiers": [], 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "tier_of": -1, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "read_tier": -1, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "write_tier": -1, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_mode": "none", 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_bytes": 0, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "target_max_objects": 0, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-13T05:03:46.390 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_flush_age": 0, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "cache_min_evict_age": 0, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profile": "", 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_params": { 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "none" 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_period": 0, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_count": 0, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "use_gmt_hitset": true, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "min_read_recency_for_promote": 0, 2024-09-13T05:03:46.391 INFO:teuthology.orchestra.run.smithi053.stdout: "min_write_recency_for_promote": 0, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_grade_decay_rate": 0, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "hit_set_search_last_n": 0, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "grade_table": [], 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "stripe_width": 0, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "expected_num_objects": 0, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "fast_read": false, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "options": {}, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "application_metadata": { 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "rbd": {} 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "read_balance": { 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "score_acting": 1.75, 2024-09-13T05:03:46.392 INFO:teuthology.orchestra.run.smithi053.stdout: "score_stable": 1.75, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "optimal_score": 1.1399999856948853, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_acting": 2, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "raw_score_stable": 2, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity_weighted": 0.79019355773925781, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity": 0.89999997615814209, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "average_primary_affinity_weighted": 0.79000002145767212 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osds": [ 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:03:46.393 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "baf16a5c-4204-424b-8d8e-a5b1486aa052", 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.394 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6816", 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6817", 2024-09-13T05:03:46.395 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6818", 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6819", 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.397 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6822", 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6823", 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6820", 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6821", 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1769628429 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.400 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6817/1769628429", 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6819/1769628429", 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6823/1769628429", 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6821/1769628429", 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:03:46.401 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f3df9a0f-3a9e-4665-9233-718bc0fe1223", 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.402 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6800", 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.403 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6801", 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6802", 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6803", 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6806", 2024-09-13T05:03:46.405 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6807", 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.406 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6804", 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6805", 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 827277153 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.407 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.408 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6801/827277153", 2024-09-13T05:03:46.408 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6803/827277153", 2024-09-13T05:03:46.408 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6807/827277153", 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6805/827277153", 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.409 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1304a876-75c5-454b-a60b-8c9823c4aca8", 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 55, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.410 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6808", 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6809", 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.411 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6810", 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.412 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6811", 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.413 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6814", 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6815", 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.414 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6812", 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6813", 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 1334123504 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.415 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6809/1334123504", 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6811/1334123504", 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6815/1334123504", 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6813/1334123504", 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "83668012-8ddb-4fe5-bb31-6e78d2723f12", 2024-09-13T05:03:46.416 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.417 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6824", 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6825", 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.418 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6826", 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6827", 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.419 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6830", 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6831", 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.420 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6828", 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.421 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.53:6829", 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 4134389313 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.53:6825/4134389313", 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.53:6827/4134389313", 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.53:6831/4134389313", 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.53:6829/4134389313", 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.422 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "f462232f-5144-4652-9139-9039d53e191a", 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:46.423 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 95, 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6808", 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.424 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6809", 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.425 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6810", 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6811", 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.426 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6815", 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6816", 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.427 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6812", 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.428 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6813", 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 3681797816 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6809/3681797816", 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6811/3681797816", 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6816/3681797816", 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6813/3681797816", 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.429 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "1d2d8607-f1d9-45e3-8486-3f5bb23b516b", 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 1, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.430 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 75, 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6800", 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.431 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6801", 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.432 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6802", 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6803", 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.433 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6806", 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6807", 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.434 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6804", 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6805", 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 81482652 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.435 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6801/81482652", 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6803/81482652", 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6807/81482652", 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6805/81482652", 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:03:46.436 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "7fe0fb7d-0a40-4666-82a1-4e18f53b445c", 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 0, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.437 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6814", 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6817", 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.438 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6818", 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6819", 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.439 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6822", 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6823", 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.440 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6820", 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6821", 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 576600432 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.441 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6817/576600432", 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6819/576600432", 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6823/576600432", 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6821/576600432", 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "uuid": "811503be-abe3-4d64-a094-09ff675f3666", 2024-09-13T05:03:46.442 INFO:teuthology.orchestra.run.smithi053.stdout: "up": 1, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "in": 1, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "weight": 1, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_affinity": 0.3215484619140625, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_begin": 0, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "last_clean_end": 0, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "up_from": 13, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "up_thru": 19, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "down_at": 0, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "lost_at": 0, 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addrs": { 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.443 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6824", 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6825", 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addrs": { 2024-09-13T05:03:46.444 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6826", 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6827", 2024-09-13T05:03:46.445 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addrs": { 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6830", 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.446 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6831", 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addrs": { 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "addrvec": [ 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v2", 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6828", 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.447 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "type": "v1", 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "addr": "172.21.15.110:6829", 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "nonce": 896779674 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "public_addr": "172.21.15.110:6825/896779674", 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "cluster_addr": "172.21.15.110:6827/896779674", 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_back_addr": "172.21.15.110:6831/896779674", 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "heartbeat_front_addr": "172.21.15.110:6829/896779674", 2024-09-13T05:03:46.448 INFO:teuthology.orchestra.run.smithi053.stdout: "state": [ 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "exists", 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "up" 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "osd_xinfo": [ 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 0, 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.449 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358655+0000", 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 1, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.343203+0000", 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.450 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 2, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.307081+0000", 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 3, 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.451 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.315153+0000", 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 4, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.452 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.280902+0000", 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 5, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.302500+0000", 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.453 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 6, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.354685+0000", 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "osd": 7, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "down_stamp": "0.000000", 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_probability": 0, 2024-09-13T05:03:46.454 INFO:teuthology.orchestra.run.smithi053.stdout: "laggy_interval": 0, 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "features": 4540138322906710015, 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "old_weight": 0, 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "last_purged_snaps_scrub": "2024-09-13T05:01:52.358223+0000", 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "dead_epoch": 0 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap": [], 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_items": [], 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_upmap_primaries": [], 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "pg_temp": [], 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "primary_temp": [], 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "blocklist": {}, 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "range_blocklist": {}, 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "erasure_code_profiles": { 2024-09-13T05:03:46.455 INFO:teuthology.orchestra.run.smithi053.stdout: "default": { 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "crush-failure-domain": "osd", 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "k": "2", 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "m": "1", 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "plugin": "jerasure", 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "technique": "reed_sol_van" 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "removed_snaps_queue": [ 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 8, 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 5 2024-09-13T05:03:46.456 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: "new_removed_snaps": [ 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: "pool": 3, 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: "snaps": [ 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: "begin": 8, 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: "length": 1 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.457 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "new_purged_snaps": [], 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "crush_node_flags": {}, 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "device_class_flags": {}, 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode": { 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_enabled": false, 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_bucket_count": 0, 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "degraded_stretch_mode": 0, 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "recovering_stretch_mode": 0, 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: "stretch_mode_bucket": 0 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.458 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:46.460 INFO:tasks.thrashosds.thrasher:j is {'epoch': 102, 'fsid': '01e1476b-1467-4e0e-b01c-0f8cdf8276fe', 'created': '2024-09-13T05:01:45.075461+0000', 'modified': '2024-09-13T05:03:45.946930+0000', 'last_up_change': '2024-09-13T05:01:55.160352+0000', 'last_in_change': '2024-09-13T05:01:47.179750+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-13T05:01:55.817858+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.7901935577392578, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-13T05:01:55.981425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7901935577392578, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-13T05:02:05.508713+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 75, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.7901935577392578, 'average_primary_affinity': 0.8999999761581421, 'average_primary_affinity_weighted': 0.7900000214576721}}], 'osds': [{'osd': 0, 'uuid': 'baf16a5c-4204-424b-8d8e-a5b1486aa052', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6816', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6817', 'nonce': 1769628429}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6818', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6819', 'nonce': 1769628429}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6822', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6823', 'nonce': 1769628429}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6820', 'nonce': 1769628429}, {'type': 'v1', 'addr': '172.21.15.53:6821', 'nonce': 1769628429}]}, 'public_addr': '172.21.15.53:6817/1769628429', 'cluster_addr': '172.21.15.53:6819/1769628429', 'heartbeat_back_addr': '172.21.15.53:6823/1769628429', 'heartbeat_front_addr': '172.21.15.53:6821/1769628429', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'f3df9a0f-3a9e-4665-9233-718bc0fe1223', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6800', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6801', 'nonce': 827277153}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6802', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6803', 'nonce': 827277153}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6806', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6807', 'nonce': 827277153}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6804', 'nonce': 827277153}, {'type': 'v1', 'addr': '172.21.15.53:6805', 'nonce': 827277153}]}, 'public_addr': '172.21.15.53:6801/827277153', 'cluster_addr': '172.21.15.53:6803/827277153', 'heartbeat_back_addr': '172.21.15.53:6807/827277153', 'heartbeat_front_addr': '172.21.15.53:6805/827277153', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '1304a876-75c5-454b-a60b-8c9823c4aca8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6808', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6809', 'nonce': 1334123504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6810', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6811', 'nonce': 1334123504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6814', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6815', 'nonce': 1334123504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6812', 'nonce': 1334123504}, {'type': 'v1', 'addr': '172.21.15.53:6813', 'nonce': 1334123504}]}, 'public_addr': '172.21.15.53:6809/1334123504', 'cluster_addr': '172.21.15.53:6811/1334123504', 'heartbeat_back_addr': '172.21.15.53:6815/1334123504', 'heartbeat_front_addr': '172.21.15.53:6813/1334123504', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '83668012-8ddb-4fe5-bb31-6e78d2723f12', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6824', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6825', 'nonce': 4134389313}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6826', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6827', 'nonce': 4134389313}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6830', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6831', 'nonce': 4134389313}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.53:6828', 'nonce': 4134389313}, {'type': 'v1', 'addr': '172.21.15.53:6829', 'nonce': 4134389313}]}, 'public_addr': '172.21.15.53:6825/4134389313', 'cluster_addr': '172.21.15.53:6827/4134389313', 'heartbeat_back_addr': '172.21.15.53:6831/4134389313', 'heartbeat_front_addr': '172.21.15.53:6829/4134389313', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': 'f462232f-5144-4652-9139-9039d53e191a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6808', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6809', 'nonce': 3681797816}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6810', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6811', 'nonce': 3681797816}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6815', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6816', 'nonce': 3681797816}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6812', 'nonce': 3681797816}, {'type': 'v1', 'addr': '172.21.15.110:6813', 'nonce': 3681797816}]}, 'public_addr': '172.21.15.110:6809/3681797816', 'cluster_addr': '172.21.15.110:6811/3681797816', 'heartbeat_back_addr': '172.21.15.110:6816/3681797816', 'heartbeat_front_addr': '172.21.15.110:6813/3681797816', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1d2d8607-f1d9-45e3-8486-3f5bb23b516b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 75, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6800', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6801', 'nonce': 81482652}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6802', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6803', 'nonce': 81482652}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6806', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6807', 'nonce': 81482652}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6804', 'nonce': 81482652}, {'type': 'v1', 'addr': '172.21.15.110:6805', 'nonce': 81482652}]}, 'public_addr': '172.21.15.110:6801/81482652', 'cluster_addr': '172.21.15.110:6803/81482652', 'heartbeat_back_addr': '172.21.15.110:6807/81482652', 'heartbeat_front_addr': '172.21.15.110:6805/81482652', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '7fe0fb7d-0a40-4666-82a1-4e18f53b445c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6814', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6817', 'nonce': 576600432}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6818', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6819', 'nonce': 576600432}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6822', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6823', 'nonce': 576600432}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6820', 'nonce': 576600432}, {'type': 'v1', 'addr': '172.21.15.110:6821', 'nonce': 576600432}]}, 'public_addr': '172.21.15.110:6817/576600432', 'cluster_addr': '172.21.15.110:6819/576600432', 'heartbeat_back_addr': '172.21.15.110:6823/576600432', 'heartbeat_front_addr': '172.21.15.110:6821/576600432', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '811503be-abe3-4d64-a094-09ff675f3666', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.3215484619140625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6824', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6825', 'nonce': 896779674}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6826', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6827', 'nonce': 896779674}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6830', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6831', 'nonce': 896779674}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6828', 'nonce': 896779674}, {'type': 'v1', 'addr': '172.21.15.110:6829', 'nonce': 896779674}]}, 'public_addr': '172.21.15.110:6825/896779674', 'cluster_addr': '172.21.15.110:6827/896779674', 'heartbeat_back_addr': '172.21.15.110:6831/896779674', 'heartbeat_front_addr': '172.21.15.110:6829/896779674', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358655+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.343203+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.307081+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.315153+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.280902+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.302500+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.354685+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-13T05:01:52.358223+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 8, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 8, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-13T05:03:46.461 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:03:46.461 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:46.462 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:46.462 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:46.462 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:46.463 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:46.459+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:46.561 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:46.560+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:46.608 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:46.608 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:46.608 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:46.608 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:46.608 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:46.618 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:46.662 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:46.661+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:46.762 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:46.762+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.766 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109393483000005, 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.771 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109377486, 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.772 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:03:46.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109360690000003, 2024-09-13T05:03:46.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:03:46.780 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109343717000002, 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.781 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:46.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.783 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:03:46.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:46.785 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109327323000002, 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:03:46.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:03:46.789 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109310287, 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.790 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.792 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:03:46.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109276703999996, 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:03:46.794 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109257575000001, 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.799 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:03:46.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109179706000006, 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:03:46.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.807 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109118960999993, 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:03:46.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:03:46.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:03:46.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109058747000006, 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.812 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:03:46.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.816 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109041419999997, 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:03:46.820 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.109012247999999, 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:46.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:03:46.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:03:46.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108992564000005, 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.826 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:03:46.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.830 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108975130999994, 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.831 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:03:46.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.835 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108957489000005, 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.836 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.840 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108923313000005, 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:46.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108897963000004, 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.845 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.846 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:03:46.849 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108880290000002, 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:03:46.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 75.108856086000003, 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:46.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:03:46.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:03:46.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:46.859 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:46.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:46.863 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:03:46.863 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:03:46.865 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-13T05:03:45.956862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":227328604,"num_objects":770,"num_object_clones":153,"num_object_copies":1540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":467,"num_read":146229,"num_read_kb":705366,"num_write":23449,"num_write_kb":383989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19480,"ondisk_log_size":19480,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418908,"kb_used_data":14916,"kb_used_omap":12,"kb_used_meta":403827,"kb_avail":754555812,"statfs":{"total":773094113280,"available":772665151488,"internally_reserved":0,"allocated":15273984,"data_stored":31525247,"data_compressed":1046277,"data_compressed_allocated":3624960,"data_compressed_original":25474908,"omap_allocated":12743,"internal_metadata":413519417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000591"},"pg_stats":[{"pgid":"3.1f","version":"97'327","reported_seq":2408,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.931786+0000","last_change":"2024-09-13T05:03:40.931786+0000","last_active":"2024-09-13T05:03:40.931786+0000","last_peered":"2024-09-13T05:03:40.931786+0000","last_clean":"2024-09-13T05:03:40.931786+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:03:40.931786+0000","last_undegraded":"2024-09-13T05:03:40.931786+0000","last_fullsized":"2024-09-13T05:03:40.931786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083326500000000003,"stat_sum":{"num_bytes":1282064,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":18,"num_read":2260,"num_read_kb":12556,"num_write":285,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.1e","version":"96'533","reported_seq":2774,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.968829+0000","last_change":"2024-09-13T05:03:40.969089+0000","last_active":"2024-09-13T05:03:40.968829+0000","last_peered":"2024-09-13T05:03:40.968829+0000","last_clean":"2024-09-13T05:03:40.968829+0000","last_became_active":"2024-09-13T05:02:06.884063+0000","last_became_peered":"2024-09-13T05:02:06.884063+0000","last_unstale":"2024-09-13T05:03:40.968829+0000","last_undegraded":"2024-09-13T05:03:40.968829+0000","last_fullsized":"2024-09-13T05:03:40.968829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":533,"log_dups_size":0,"ondisk_log_size":533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:54:39.216716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068329199999999995,"stat_sum":{"num_bytes":3334252,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":10,"num_read":3134,"num_read_kb":18928,"num_write":632,"num_write_kb":17867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.1d","version":"94'483","reported_seq":2493,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.943063+0000","last_change":"2024-09-13T05:03:40.943421+0000","last_active":"2024-09-13T05:03:40.943063+0000","last_peered":"2024-09-13T05:03:40.943063+0000","last_clean":"2024-09-13T05:03:40.943063+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:03:40.943063+0000","last_undegraded":"2024-09-13T05:03:40.943063+0000","last_fullsized":"2024-09-13T05:03:40.943063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076572900000000004,"stat_sum":{"num_bytes":839680,"num_objects":15,"num_object_clones":3,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":3678,"num_read_kb":7593,"num_write":571,"num_write_kb":3099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.1c","version":"97'571","reported_seq":5359,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.006790+0000","last_change":"2024-09-13T05:03:40.943485+0000","last_active":"2024-09-13T05:03:41.006790+0000","last_peered":"2024-09-13T05:03:41.006790+0000","last_clean":"2024-09-13T05:03:41.006790+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:03:41.006790+0000","last_undegraded":"2024-09-13T05:03:41.006790+0000","last_fullsized":"2024-09-13T05:03:41.006790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":571,"log_dups_size":0,"ondisk_log_size":571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076388599999999997,"stat_sum":{"num_bytes":5264850,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":12,"num_read":6277,"num_read_kb":35014,"num_write":690,"num_write_kb":21058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.1b","version":"96'708","reported_seq":5454,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.933201+0000","last_change":"2024-09-13T05:03:40.933379+0000","last_active":"2024-09-13T05:03:40.933201+0000","last_peered":"2024-09-13T05:03:40.933201+0000","last_clean":"2024-09-13T05:03:40.933201+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:03:40.933201+0000","last_undegraded":"2024-09-13T05:03:40.933201+0000","last_fullsized":"2024-09-13T05:03:40.933201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":708,"log_dups_size":0,"ondisk_log_size":708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021268820000000001,"stat_sum":{"num_bytes":12473028,"num_objects":34,"num_object_clones":6,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":7592,"num_read_kb":27519,"num_write":688,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.1a","version":"96'1762","reported_seq":6199,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.038151+0000","last_change":"2024-09-13T05:03:40.948149+0000","last_active":"2024-09-13T05:03:41.038151+0000","last_peered":"2024-09-13T05:03:41.038151+0000","last_clean":"2024-09-13T05:03:41.038151+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:03:41.038151+0000","last_undegraded":"2024-09-13T05:03:41.038151+0000","last_fullsized":"2024-09-13T05:03:41.038151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1762,"log_dups_size":0,"ondisk_log_size":1762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010442019999999999,"stat_sum":{"num_bytes":11023823,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":19,"num_read":12285,"num_read_kb":32048,"num_write":2854,"num_write_kb":15669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.19","version":"94'458","reported_seq":1580,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.934109+0000","last_change":"2024-09-13T05:03:40.934356+0000","last_active":"2024-09-13T05:03:40.934109+0000","last_peered":"2024-09-13T05:03:40.934109+0000","last_clean":"2024-09-13T05:03:40.934109+0000","last_became_active":"2024-09-13T05:02:06.881244+0000","last_became_peered":"2024-09-13T05:02:06.881244+0000","last_unstale":"2024-09-13T05:03:40.934109+0000","last_undegraded":"2024-09-13T05:03:40.934109+0000","last_fullsized":"2024-09-13T05:03:40.934109+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:05:21.706328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034620199999999999,"stat_sum":{"num_bytes":819200,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":10,"num_read":2464,"num_read_kb":10969,"num_write":530,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.18","version":"97'385","reported_seq":2352,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.968745+0000","last_change":"2024-09-13T05:03:40.968745+0000","last_active":"2024-09-13T05:03:40.968745+0000","last_peered":"2024-09-13T05:03:40.968745+0000","last_clean":"2024-09-13T05:03:40.968745+0000","last_became_active":"2024-09-13T05:02:06.880998+0000","last_became_peered":"2024-09-13T05:02:06.880998+0000","last_unstale":"2024-09-13T05:03:40.968745+0000","last_undegraded":"2024-09-13T05:03:40.968745+0000","last_fullsized":"2024-09-13T05:03:40.968745+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:51:16.614368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040301999999999997,"stat_sum":{"num_bytes":8732672,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":14,"num_read":2298,"num_read_kb":13275,"num_write":478,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.17","version":"96'375","reported_seq":2708,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.932863+0000","last_change":"2024-09-13T05:03:40.933281+0000","last_active":"2024-09-13T05:03:40.932863+0000","last_peered":"2024-09-13T05:03:40.932863+0000","last_clean":"2024-09-13T05:03:40.932863+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:03:40.932863+0000","last_undegraded":"2024-09-13T05:03:40.932863+0000","last_fullsized":"2024-09-13T05:03:40.932863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021542050000000002,"stat_sum":{"num_bytes":12651348,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":2745,"num_read_kb":13827,"num_write":376,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.16","version":"96'347","reported_seq":3191,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.943017+0000","last_change":"2024-09-13T05:03:40.943200+0000","last_active":"2024-09-13T05:03:40.943017+0000","last_peered":"2024-09-13T05:03:40.943017+0000","last_clean":"2024-09-13T05:03:40.943017+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:03:40.943017+0000","last_undegraded":"2024-09-13T05:03:40.943017+0000","last_fullsized":"2024-09-13T05:03:40.943017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":347,"log_dups_size":0,"ondisk_log_size":347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056572299999999996,"stat_sum":{"num_bytes":4174483,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":10,"num_read":3239,"num_read_kb":17273,"num_write":312,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.15","version":"95'618","reported_seq":3582,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.969154+0000","last_change":"2024-09-13T05:03:40.969363+0000","last_active":"2024-09-13T05:03:40.969154+0000","last_peered":"2024-09-13T05:03:40.969154+0000","last_clean":"2024-09-13T05:03:40.969154+0000","last_became_active":"2024-09-13T05:03:19.880268+0000","last_became_peered":"2024-09-13T05:03:19.880268+0000","last_unstale":"2024-09-13T05:03:40.969154+0000","last_undegraded":"2024-09-13T05:03:40.969154+0000","last_fullsized":"2024-09-13T05:03:40.969154+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:15:37.528920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084514900000000001,"stat_sum":{"num_bytes":6143455,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":15,"num_read":4194,"num_read_kb":19459,"num_write":834,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.14","version":"94'336","reported_seq":2051,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.946994+0000","last_change":"2024-09-13T05:03:40.947218+0000","last_active":"2024-09-13T05:03:40.946994+0000","last_peered":"2024-09-13T05:03:40.946994+0000","last_clean":"2024-09-13T05:03:40.946994+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:03:40.946994+0000","last_undegraded":"2024-09-13T05:03:40.946994+0000","last_fullsized":"2024-09-13T05:03:40.946994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039169900000000002,"stat_sum":{"num_bytes":4618141,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":12,"num_read":1612,"num_read_kb":13794,"num_write":324,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.7","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.998725+0000","last_change":"2024-09-13T05:03:08.934184+0000","last_active":"2024-09-13T05:03:39.998725+0000","last_peered":"2024-09-13T05:03:39.998725+0000","last_clean":"2024-09-13T05:03:39.998725+0000","last_became_active":"2024-09-13T05:03:08.933851+0000","last_became_peered":"2024-09-13T05:03:08.933851+0000","last_unstale":"2024-09-13T05:03:39.998725+0000","last_undegraded":"2024-09-13T05:03:39.998725+0000","last_fullsized":"2024-09-13T05:03:39.998725+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:30:14.709040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.5","version":"96'452","reported_seq":2458,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.943536+0000","last_change":"2024-09-13T05:03:40.943726+0000","last_active":"2024-09-13T05:03:40.943536+0000","last_peered":"2024-09-13T05:03:40.943536+0000","last_clean":"2024-09-13T05:03:40.943536+0000","last_became_active":"2024-09-13T05:02:06.881356+0000","last_became_peered":"2024-09-13T05:02:06.881356+0000","last_unstale":"2024-09-13T05:03:40.943536+0000","last_undegraded":"2024-09-13T05:03:40.943536+0000","last_fullsized":"2024-09-13T05:03:40.943536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:52:37.032892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00100534,"stat_sum":{"num_bytes":10537488,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":2567,"num_read_kb":11151,"num_write":408,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.6","version":"0'0","reported_seq":42,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.924074+0000","last_change":"2024-09-13T05:03:08.934462+0000","last_active":"2024-09-13T05:03:39.924074+0000","last_peered":"2024-09-13T05:03:39.924074+0000","last_clean":"2024-09-13T05:03:39.924074+0000","last_became_active":"2024-09-13T05:03:08.934091+0000","last_became_peered":"2024-09-13T05:03:08.934091+0000","last_unstale":"2024-09-13T05:03:39.924074+0000","last_undegraded":"2024-09-13T05:03:39.924074+0000","last_fullsized":"2024-09-13T05:03:39.924074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:06:44.872022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"97'1060","reported_seq":4724,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.051508+0000","last_change":"2024-09-13T05:03:40.934641+0000","last_active":"2024-09-13T05:03:41.051508+0000","last_peered":"2024-09-13T05:03:41.051508+0000","last_clean":"2024-09-13T05:03:41.051508+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:03:41.051508+0000","last_undegraded":"2024-09-13T05:03:41.051508+0000","last_fullsized":"2024-09-13T05:03:41.051508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1060,"log_dups_size":0,"ondisk_log_size":1060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033295999999999998,"stat_sum":{"num_bytes":8560551,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":21,"num_read":8326,"num_read_kb":32945,"num_write":1306,"num_write_kb":8926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.925729+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:03:39.925729+0000","last_peered":"2024-09-13T05:03:39.925729+0000","last_clean":"2024-09-13T05:03:39.925729+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:03:39.925729+0000","last_undegraded":"2024-09-13T05:03:39.925729+0000","last_fullsized":"2024-09-13T05:03:39.925729+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"96'746","reported_seq":2430,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.968878+0000","last_change":"2024-09-13T05:03:40.969083+0000","last_active":"2024-09-13T05:03:40.968878+0000","last_peered":"2024-09-13T05:03:40.968878+0000","last_clean":"2024-09-13T05:03:40.968878+0000","last_became_active":"2024-09-13T05:02:06.884640+0000","last_became_peered":"2024-09-13T05:02:06.884640+0000","last_unstale":"2024-09-13T05:03:40.968878+0000","last_undegraded":"2024-09-13T05:03:40.968878+0000","last_fullsized":"2024-09-13T05:03:40.968878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":746,"log_dups_size":0,"ondisk_log_size":746,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:24:51.793989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056217599999999995,"stat_sum":{"num_bytes":3376357,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":21,"num_read":3800,"num_read_kb":26986,"num_write":875,"num_write_kb":24239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.998767+0000","last_change":"2024-09-13T05:01:59.564904+0000","last_active":"2024-09-13T05:03:39.998767+0000","last_peered":"2024-09-13T05:03:39.998767+0000","last_clean":"2024-09-13T05:03:39.998767+0000","last_became_active":"2024-09-13T05:01:57.175607+0000","last_became_peered":"2024-09-13T05:01:57.175607+0000","last_unstale":"2024-09-13T05:03:39.998767+0000","last_undegraded":"2024-09-13T05:03:39.998767+0000","last_fullsized":"2024-09-13T05:03:39.998767+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:18:33.330452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029124199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"97'689","reported_seq":3367,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.018872+0000","last_change":"2024-09-13T05:03:40.933020+0000","last_active":"2024-09-13T05:03:41.018872+0000","last_peered":"2024-09-13T05:03:41.018872+0000","last_clean":"2024-09-13T05:03:41.018872+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:03:41.018872+0000","last_undegraded":"2024-09-13T05:03:41.018872+0000","last_fullsized":"2024-09-13T05:03:41.018872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":689,"log_dups_size":0,"ondisk_log_size":689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001947252,"stat_sum":{"num_bytes":5937863,"num_objects":22,"num_object_clones":5,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":12,"num_read":4741,"num_read_kb":21460,"num_write":773,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.1","version":"96'422","reported_seq":2393,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.931125+0000","last_change":"2024-09-13T05:03:40.931414+0000","last_active":"2024-09-13T05:03:40.931125+0000","last_peered":"2024-09-13T05:03:40.931125+0000","last_clean":"2024-09-13T05:03:40.931125+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:03:40.931125+0000","last_undegraded":"2024-09-13T05:03:40.931125+0000","last_fullsized":"2024-09-13T05:03:40.931125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":422,"log_dups_size":0,"ondisk_log_size":422,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047017199999999998,"stat_sum":{"num_bytes":8443062,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":2669,"num_read_kb":22771,"num_write":476,"num_write_kb":11861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"2.0","version":"16'117","reported_seq":189,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.928562+0000","last_change":"2024-09-13T05:03:19.879930+0000","last_active":"2024-09-13T05:03:39.928562+0000","last_peered":"2024-09-13T05:03:39.928562+0000","last_clean":"2024-09-13T05:03:39.928562+0000","last_became_active":"2024-09-13T05:03:19.879567+0000","last_became_peered":"2024-09-13T05:03:19.879567+0000","last_unstale":"2024-09-13T05:03:39.928562+0000","last_undegraded":"2024-09-13T05:03:39.928562+0000","last_fullsized":"2024-09-13T05:03:39.928562+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:10:32.328565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":97,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.928542+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:03:39.928542+0000","last_peered":"2024-09-13T05:03:39.928542+0000","last_clean":"2024-09-13T05:03:39.928542+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:03:39.928542+0000","last_undegraded":"2024-09-13T05:03:39.928542+0000","last_fullsized":"2024-09-13T05:03:39.928542+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"97'557","reported_seq":4024,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.985073+0000","last_change":"2024-09-13T05:03:40.934589+0000","last_active":"2024-09-13T05:03:40.985073+0000","last_peered":"2024-09-13T05:03:40.985073+0000","last_clean":"2024-09-13T05:03:40.985073+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:03:40.985073+0000","last_undegraded":"2024-09-13T05:03:40.985073+0000","last_fullsized":"2024-09-13T05:03:40.985073+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039656199999999999,"stat_sum":{"num_bytes":8417554,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":25,"num_read":5749,"num_read_kb":38805,"num_write":564,"num_write_kb":13172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.2","version":"17'2","reported_seq":46,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.923981+0000","last_change":"2024-09-13T05:03:19.881032+0000","last_active":"2024-09-13T05:03:39.923981+0000","last_peered":"2024-09-13T05:03:39.923981+0000","last_clean":"2024-09-13T05:03:39.923981+0000","last_became_active":"2024-09-13T05:03:19.880757+0000","last_became_peered":"2024-09-13T05:03:19.880757+0000","last_unstale":"2024-09-13T05:03:39.923981+0000","last_undegraded":"2024-09-13T05:03:39.923981+0000","last_fullsized":"2024-09-13T05:03:39.923981+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:32:37.512669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"97'561","reported_seq":3300,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.052164+0000","last_change":"2024-09-13T05:03:40.947844+0000","last_active":"2024-09-13T05:03:41.052164+0000","last_peered":"2024-09-13T05:03:41.052164+0000","last_clean":"2024-09-13T05:03:41.052164+0000","last_became_active":"2024-09-13T05:02:06.880673+0000","last_became_peered":"2024-09-13T05:02:06.880673+0000","last_unstale":"2024-09-13T05:03:41.052164+0000","last_undegraded":"2024-09-13T05:03:41.052164+0000","last_fullsized":"2024-09-13T05:03:41.052164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":561,"log_dups_size":0,"ondisk_log_size":561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:45:24.929086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077937700000000004,"stat_sum":{"num_bytes":18428982,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":3839,"num_read_kb":35322,"num_write":525,"num_write_kb":11875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.1","version":"0'0","reported_seq":96,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.931269+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:03:39.931269+0000","last_peered":"2024-09-13T05:03:39.931269+0000","last_clean":"2024-09-13T05:03:39.931269+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:03:39.931269+0000","last_undegraded":"2024-09-13T05:03:39.931269+0000","last_fullsized":"2024-09-13T05:03:39.931269+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"94'1161","reported_seq":3547,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.968543+0000","last_change":"2024-09-13T05:03:40.968739+0000","last_active":"2024-09-13T05:03:40.968543+0000","last_peered":"2024-09-13T05:03:40.968543+0000","last_clean":"2024-09-13T05:03:40.968543+0000","last_became_active":"2024-09-13T05:02:06.882765+0000","last_became_peered":"2024-09-13T05:02:06.882765+0000","last_unstale":"2024-09-13T05:03:40.968543+0000","last_undegraded":"2024-09-13T05:03:40.968543+0000","last_fullsized":"2024-09-13T05:03:40.968543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1161,"log_dups_size":0,"ondisk_log_size":1161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:21.053116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032779199999999998,"stat_sum":{"num_bytes":1251548,"num_objects":22,"num_object_clones":2,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":7480,"num_read_kb":16372,"num_write":1803,"num_write_kb":6231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":96,"state":"active+clean","last_fresh":"2024-09-13T05:03:39.923901+0000","last_change":"2024-09-13T05:03:19.881102+0000","last_active":"2024-09-13T05:03:39.923901+0000","last_peered":"2024-09-13T05:03:39.923901+0000","last_clean":"2024-09-13T05:03:39.923901+0000","last_became_active":"2024-09-13T05:03:19.880843+0000","last_became_peered":"2024-09-13T05:03:19.880843+0000","last_unstale":"2024-09-13T05:03:39.923901+0000","last_undegraded":"2024-09-13T05:03:39.923901+0000","last_fullsized":"2024-09-13T05:03:39.923901+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:02:23.081525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"96'426","reported_seq":3342,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.969210+0000","last_change":"2024-09-13T05:03:40.969459+0000","last_active":"2024-09-13T05:03:40.969210+0000","last_peered":"2024-09-13T05:03:40.969210+0000","last_clean":"2024-09-13T05:03:40.969210+0000","last_became_active":"2024-09-13T05:02:06.881316+0000","last_became_peered":"2024-09-13T05:02:06.881316+0000","last_unstale":"2024-09-13T05:03:40.969210+0000","last_undegraded":"2024-09-13T05:03:40.969210+0000","last_fullsized":"2024-09-13T05:03:40.969210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":426,"log_dups_size":0,"ondisk_log_size":426,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:33:07.292949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092503300000000002,"stat_sum":{"num_bytes":3975350,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":3044,"num_read_kb":25130,"num_write":469,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.9","version":"97'441","reported_seq":2776,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.049591+0000","last_change":"2024-09-13T05:03:40.947723+0000","last_active":"2024-09-13T05:03:41.049591+0000","last_peered":"2024-09-13T05:03:41.049591+0000","last_clean":"2024-09-13T05:03:41.049591+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:03:41.049591+0000","last_undegraded":"2024-09-13T05:03:41.049591+0000","last_fullsized":"2024-09-13T05:03:41.049591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079751500000000005,"stat_sum":{"num_bytes":5685716,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":11,"num_read":2756,"num_read_kb":23076,"num_write":508,"num_write_kb":9845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.a","version":"97'617","reported_seq":4574,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.947979+0000","last_change":"2024-09-13T05:03:40.947979+0000","last_active":"2024-09-13T05:03:40.947979+0000","last_peered":"2024-09-13T05:03:40.947979+0000","last_clean":"2024-09-13T05:03:40.947979+0000","last_became_active":"2024-09-13T05:03:08.934100+0000","last_became_peered":"2024-09-13T05:03:08.934100+0000","last_unstale":"2024-09-13T05:03:40.947979+0000","last_undegraded":"2024-09-13T05:03:40.947979+0000","last_fullsized":"2024-09-13T05:03:40.947979+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:53:15.659132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00089233599999999998,"stat_sum":{"num_bytes":1574492,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":20,"num_read":4965,"num_read_kb":15923,"num_write":652,"num_write_kb":24384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.b","version":"97'872","reported_seq":2507,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.969611+0000","last_change":"2024-09-13T05:03:40.969611+0000","last_active":"2024-09-13T05:03:40.969611+0000","last_peered":"2024-09-13T05:03:40.969611+0000","last_clean":"2024-09-13T05:03:40.969611+0000","last_became_active":"2024-09-13T05:02:06.881094+0000","last_became_peered":"2024-09-13T05:02:06.881094+0000","last_unstale":"2024-09-13T05:03:40.969611+0000","last_undegraded":"2024-09-13T05:03:40.969611+0000","last_fullsized":"2024-09-13T05:03:40.969611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":872,"log_dups_size":0,"ondisk_log_size":872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:02:56.840152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092978400000000001,"stat_sum":{"num_bytes":2691270,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":4682,"num_read_kb":9579,"num_write":1229,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.c","version":"96'407","reported_seq":3073,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.942668+0000","last_change":"2024-09-13T05:03:40.942982+0000","last_active":"2024-09-13T05:03:40.942668+0000","last_peered":"2024-09-13T05:03:40.942668+0000","last_clean":"2024-09-13T05:03:40.942668+0000","last_became_active":"2024-09-13T05:02:06.879695+0000","last_became_peered":"2024-09-13T05:02:06.879695+0000","last_unstale":"2024-09-13T05:03:40.942668+0000","last_undegraded":"2024-09-13T05:03:40.942668+0000","last_fullsized":"2024-09-13T05:03:40.942668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:42:23.915465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040159899999999999,"stat_sum":{"num_bytes":1294376,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":15,"num_read":3536,"num_read_kb":10428,"num_write":352,"num_write_kb":2864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.d","version":"97'556","reported_seq":5231,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.005969+0000","last_change":"2024-09-13T05:03:40.942977+0000","last_active":"2024-09-13T05:03:41.005969+0000","last_peered":"2024-09-13T05:03:41.005969+0000","last_clean":"2024-09-13T05:03:41.005969+0000","last_became_active":"2024-09-13T05:03:19.880315+0000","last_became_peered":"2024-09-13T05:03:19.880315+0000","last_unstale":"2024-09-13T05:03:41.005969+0000","last_undegraded":"2024-09-13T05:03:41.005969+0000","last_fullsized":"2024-09-13T05:03:41.005969+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:15:27.243701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037222500000000001,"stat_sum":{"num_bytes":6635287,"num_objects":23,"num_object_clones":5,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":14,"num_read":7557,"num_read_kb":36552,"num_write":605,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.e","version":"95'572","reported_seq":5547,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.946941+0000","last_change":"2024-09-13T05:03:40.947214+0000","last_active":"2024-09-13T05:03:40.946941+0000","last_peered":"2024-09-13T05:03:40.946941+0000","last_clean":"2024-09-13T05:03:40.946941+0000","last_became_active":"2024-09-13T05:03:19.880257+0000","last_became_peered":"2024-09-13T05:03:19.880257+0000","last_unstale":"2024-09-13T05:03:40.946941+0000","last_undegraded":"2024-09-13T05:03:40.946941+0000","last_fullsized":"2024-09-13T05:03:40.946941+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:07.964691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038950500000000001,"stat_sum":{"num_bytes":3647029,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":14,"num_read":6407,"num_read_kb":60503,"num_write":676,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.f","version":"96'379","reported_seq":2973,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.026212+0000","last_change":"2024-09-13T05:03:40.947565+0000","last_active":"2024-09-13T05:03:41.026212+0000","last_peered":"2024-09-13T05:03:41.026212+0000","last_clean":"2024-09-13T05:03:41.026212+0000","last_became_active":"2024-09-13T05:03:39.927143+0000","last_became_peered":"2024-09-13T05:03:39.927143+0000","last_unstale":"2024-09-13T05:03:41.026212+0000","last_undegraded":"2024-09-13T05:03:41.026212+0000","last_fullsized":"2024-09-13T05:03:41.026212+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":379,"log_dups_size":0,"ondisk_log_size":379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:19:29.447241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00068396600000000004,"stat_sum":{"num_bytes":14502773,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":11,"num_read":3115,"num_read_kb":14090,"num_write":336,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.10","version":"97'860","reported_seq":3036,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.012655+0000","last_change":"2024-09-13T05:03:40.931409+0000","last_active":"2024-09-13T05:03:41.012655+0000","last_peered":"2024-09-13T05:03:41.012655+0000","last_clean":"2024-09-13T05:03:41.012655+0000","last_became_active":"2024-09-13T05:03:08.933851+0000","last_became_peered":"2024-09-13T05:03:08.933851+0000","last_unstale":"2024-09-13T05:03:41.012655+0000","last_undegraded":"2024-09-13T05:03:41.012655+0000","last_fullsized":"2024-09-13T05:03:41.012655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":860,"log_dups_size":0,"ondisk_log_size":860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:02:10.352633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053524699999999998,"stat_sum":{"num_bytes":8162908,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":18,"num_read":5378,"num_read_kb":11761,"num_write":1167,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.11","version":"97'504","reported_seq":4069,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:41.037415+0000","last_change":"2024-09-13T05:03:40.947467+0000","last_active":"2024-09-13T05:03:41.037415+0000","last_peered":"2024-09-13T05:03:41.037415+0000","last_clean":"2024-09-13T05:03:41.037415+0000","last_became_active":"2024-09-13T05:03:19.880261+0000","last_became_peered":"2024-09-13T05:03:19.880261+0000","last_unstale":"2024-09-13T05:03:41.037415+0000","last_undegraded":"2024-09-13T05:03:41.037415+0000","last_fullsized":"2024-09-13T05:03:41.037415+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":504,"log_dups_size":0,"ondisk_log_size":504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:14.799250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059319600000000004,"stat_sum":{"num_bytes":23286251,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5023,"num_read_kb":13388,"num_write":484,"num_write_kb":2566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.12","version":"97'430","reported_seq":3666,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.931836+0000","last_change":"2024-09-13T05:03:40.931836+0000","last_active":"2024-09-13T05:03:40.931836+0000","last_peered":"2024-09-13T05:03:40.931836+0000","last_clean":"2024-09-13T05:03:40.931836+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:03:40.931836+0000","last_undegraded":"2024-09-13T05:03:40.931836+0000","last_fullsized":"2024-09-13T05:03:40.931836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081641799999999996,"stat_sum":{"num_bytes":8340128,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":13,"num_read":4139,"num_read_kb":35871,"num_write":435,"num_write_kb":18025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"a"}]},{"pgid":"3.13","version":"96'745","reported_seq":3192,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-09-13T05:03:40.930851+0000","last_change":"2024-09-13T05:03:40.931076+0000","last_active":"2024-09-13T05:03:40.930851+0000","last_peered":"2024-09-13T05:03:40.930851+0000","last_clean":"2024-09-13T05:03:40.930851+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:03:40.930851+0000","last_undegraded":"2024-09-13T05:03:40.930851+0000","last_fullsized":"2024-09-13T05:03:40.930851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":745,"log_dups_size":0,"ondisk_log_size":745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033908700000000001,"stat_sum":{"num_bytes":10632236,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":14,"num_read":4562,"num_read_kb":20898,"num_write":1071,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"d","length":"a"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":226738217,"num_objects":766,"num_object_clones":153,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":467,"num_read":146113,"num_read_kb":705266,"num_write":23288,"num_write_kb":381571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":5603328,"data_stored":25304164,"data_compressed":826604,"data_compressed_allocated":3293184,"data_compressed_original":23697244,"omap_allocated":0,"internal_metadata":0},"log_size":19360,"ondisk_log_size":19360,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55272,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316568,"statfs":{"total":96636764160,"available":96580165632,"internally_reserved":0,"allocated":2048000,"data_stored":8976858,"data_compressed":239873,"data_compressed_allocated":512000,"data_compressed_original":8159232,"omap_allocated":1589,"internal_metadata":54524363},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.308}]},{"osd":1,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.32100000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.45500000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.432}]},{"osd":5,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.249},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29499999999999998}]},{"osd":6,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.34499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40716,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331124,"statfs":{"total":96636764160,"available":96595070976,"internally_reserved":0,"allocated":1441792,"data_stored":1174874,"data_compressed":1753,"data_compressed_allocated":200704,"data_compressed_original":561152,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.51700000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.57599999999999996}]},{"osd":2,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.624},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59699999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.41299999999999998}]},{"osd":7,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.378},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.53500000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1785856,"data_stored":5644733,"data_compressed":181885,"data_compressed_allocated":385024,"data_compressed_original":4927488,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.39800000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.5}]},{"osd":2,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.56799999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.58499999999999996}]},{"osd":7,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.61299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65364,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306476,"statfs":{"total":96636764160,"available":96569831424,"internally_reserved":0,"allocated":2293760,"data_stored":3204918,"data_compressed":142910,"data_compressed_allocated":622592,"data_compressed_original":2299005,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":6.5469999999999997,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":6.4509999999999996,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.57099999999999995}]},{"osd":1,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":2.4049999999999998,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":6.5190000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.4880000000000004,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":3.3460000000000001,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":6.577,"5min":6.577,"15min":6.577},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":3.379,"5min":3.379,"15min":3.379},"last":0.34899999999999998}]},{"osd":6,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.318},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":2.371,"5min":2.371,"15min":2.371},"last":0.371}]},{"osd":7,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":6.6020000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":3.6869999999999998,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.28299999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55352,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316488,"statfs":{"total":96636764160,"available":96580083712,"internally_reserved":0,"allocated":2134016,"data_stored":4763922,"data_compressed":161438,"data_compressed_allocated":622592,"data_compressed_original":3987249,"omap_allocated":1592,"internal_metadata":54524360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.245},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.40699999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.48299999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.496}]},{"osd":5,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.8090000000000002,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.70599999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.626},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68700000000000006}]},{"osd":7,"last update":"Fri Sep 13 05:02:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":3.02,"5min":3.02,"15min":3.02},"last":0.66500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40788,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331052,"statfs":{"total":96636764160,"available":96594997248,"internally_reserved":0,"allocated":1511424,"data_stored":1238149,"data_compressed":57711,"data_compressed_allocated":221184,"data_compressed_original":595069,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.372}]},{"osd":1,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32900000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1,"5min":1,"15min":1},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.311}]},{"osd":4,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.433}]},{"osd":5,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48499999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Sep 13 05:02:56 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.495},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45000000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":73304,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298536,"statfs":{"total":96636764160,"available":96561700864,"internally_reserved":0,"allocated":2043904,"data_stored":2850358,"data_compressed":116189,"data_compressed_allocated":598016,"data_compressed_original":2098993,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"last":0.24399999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.442,"5min":0.442,"15min":0.442},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.314}]},{"osd":3,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.432}]},{"osd":5,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45200000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.40200000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:02:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.51200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316604,"statfs":{"total":96636764160,"available":96580202496,"internally_reserved":0,"allocated":2015232,"data_stored":3671435,"data_compressed":144518,"data_compressed_allocated":462848,"data_compressed_original":2846720,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.29199999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"last":0.316},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.33400000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.39400000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.44800000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.63},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.66600000000000004}]},{"osd":7,"last update":"Fri Sep 13 05:02:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.54900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":794624,"data_stored":3021399,"data_compressed":116268,"data_compressed_allocated":425984,"data_compressed_original":2752512,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":786432,"data_stored":1645023,"data_compressed":78088,"data_compressed_allocated":520192,"data_compressed_original":1460017,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":303104,"data_stored":537057,"data_compressed":24371,"data_compressed_allocated":172032,"data_compressed_original":443517,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":970752,"data_stored":4170969,"data_compressed":153674,"data_compressed_allocated":610304,"data_compressed_original":3938097,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1142784,"data_stored":2659252,"data_compressed":142910,"data_compressed_allocated":622592,"data_compressed_original":2299005,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":4905586,"data_compressed":102650,"data_compressed_allocated":299008,"data_compressed_original":4751360,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":629159,"data_compressed":1753,"data_compressed_allocated":200704,"data_compressed_original":561152,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":790528,"data_stored":7735719,"data_compressed":206890,"data_compressed_allocated":442368,"data_compressed_original":7491584,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:03:46.869 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [4, 7, 3, 5] 2024-09-13T05:03:46.869 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.3', '4', '7', '3', '5'] 2024-09-13T05:03:46.869 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.3 4 7 3 5 2024-09-13T05:03:46.870 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:46.870+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:46.909 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:03:46.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:46.992 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:46.991+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:106314 3.0 3:02b85d7b:::rbd_header.117cdd702390:head [call lock.lock in=76b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e103)", 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:47.036356+0000", 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 0.00029323199999999999, 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000311432, 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:47.037 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 106314 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036356+0000", 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.038 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036356+0000", 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036355+0000", 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036361+0000", 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6230000000000002e-06 2024-09-13T05:03:47.039 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036374+0000", 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3013000000000001e-05 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036382+0000", 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8129999999999996e-06 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:47.040 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036431+0000", 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9567000000000002e-05 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:47.036517+0000", 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5736999999999996e-05 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: ], 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 1 2024-09-13T05:03:47.041 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:47.051 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:47.073 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:47.072+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:47.145 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:47.145 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:47.170 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:47.170 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:47.170 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:47.173 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:47.173+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:47.179 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:47.179 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:47.180 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:47.209 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:47.210 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:47.227 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:47.227 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:47.244 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:47.245 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:47.261 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:47.262 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:47.274 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:47.274+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:47.277 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:47.278 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:47.295 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:47.296 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:47.322 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:47.322 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:47.331 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:47.415 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:47.415+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:47.476 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:47.475+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:47.478 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:47.478 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:47.487 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:47.577 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:47.577+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:47.634 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:47.634 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:47.643 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:47.678 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:47.678+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:47.749 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:47.749 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:47.758 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:47.780 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:47.779+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:47.865 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:47.865 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:47.874 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:47.881 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:47.880+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:47.962 INFO:teuthology.orchestra.run.smithi053.stderr:set 3.3 pg_upmap_items mapping to [4->7,3->5] 2024-09-13T05:03:47.962 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:03:47.982 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:47.981+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:47.992 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:47.992 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.002 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:48.086 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:48.086+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:48.150 INFO:teuthology.orchestra.run.smithi053.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.150 INFO:teuthology.orchestra.run.smithi053.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:48.203 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:48.202+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:48.263 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.263 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.272 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:48.314 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:48.313+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:48.386 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:48.385+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:48.414 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.414 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.423 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:48.486 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:48.486+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.564 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.573 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:48.587 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:48.587+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:48.675 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.675 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.683 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:48.728 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:48.727+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:48.786 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.786 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.789 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:48.788+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:48.794 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:48.890 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:48.890+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:48.936 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:48.936 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:48.945 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:49.030 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:49.029+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:49.088 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:49.089 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:49.092 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:49.091+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:49.097 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:49.194 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:49.193+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:49.240 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:49.240 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:49.249 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:49.294 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:49.294+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:49.310 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-13T05:03:49.391 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:49.391 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:49.397 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:49.397+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:49.400 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:49.447 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:49.447 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:49.463 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:49.464 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:49.479 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:49.479 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:49.495 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:49.495 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:49.498 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:49.498+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:49.502 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:49.502 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:49.511 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:49.512 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:49.512 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:49.528 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:49.529 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:49.545 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:49.545 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:49.562 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:49.562 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:49.614 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:49.614 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [], 2024-09-13T05:03:49.614 INFO:teuthology.orchestra.run.smithi110.stdout: "complaint_time": 30, 2024-09-13T05:03:49.614 INFO:teuthology.orchestra.run.smithi110.stdout: "num_blocked_ops": 0 2024-09-13T05:03:49.614 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:49.615 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:49.614+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:49.623 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:49.702 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:49.702+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:49.725 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "size": 20, 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 600, 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "ops": [ 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21843 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1777664~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.495427101999994, 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037774416999999998, 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.726 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21843 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228974+0000", 2024-09-13T05:03:49.727 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228805+0000", 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9998312 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228976+0000", 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00017051699999999999 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.728 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228980+0000", 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4599999999999996e-06 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.228983+0000", 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0960000000000001e-06 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266594+0000", 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037610215000000002 2024-09-13T05:03:49.729 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266666+0000", 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1989000000000005e-05 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.266749+0000", 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3052000000000004e-05 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.730 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21850 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2465792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.494498992999993, 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037825036999999999, 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21850 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.731 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229902+0000", 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229861+0000", 2024-09-13T05:03:49.732 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.999959 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229904+0000", 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2475999999999998e-05 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229908+0000", 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.8060000000000001e-06 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.733 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.229911+0000", 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7330000000000001e-06 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267644+0000", 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037732313000000003 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267680+0000", 2024-09-13T05:03:49.734 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6826999999999998e-05 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.267727+0000", 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6915000000000001e-05 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21857 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3153920~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.493400584, 2024-09-13T05:03:49.735 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037781006999999998, 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21857 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:49.736 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231001+0000", 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.737 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231002+0000", 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.3999999999999999e-06 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231005+0000", 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8550000000000002e-06 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.231007+0000", 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4150000000000002e-06 2024-09-13T05:03:49.738 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268627+0000", 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037619517999999998 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268698+0000", 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.1575000000000005e-05 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.739 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.268782+0000", 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.3462999999999997e-05 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21936 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.487340622999994, 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038795833000000002, 2024-09-13T05:03:49.740 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21936 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.741 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237061+0000", 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237060+0000", 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237062+0000", 2024-09-13T05:03:49.742 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5910000000000001e-06 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237065+0000", 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0879999999999999e-06 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237068+0000", 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.481e-06 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.743 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274252+0000", 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037184371000000001 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274300+0000", 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.782e-05 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274514+0000", 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.000214527 2024-09-13T05:03:49.744 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275735+0000", 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0012205580000000001 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275781+0000", 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.6248999999999997e-05 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.745 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275824+0000", 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.2409e-05 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275857+0000", 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.2988999999999999e-05 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21937 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 3358720~3513] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.746 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.487331628000007, 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038049144, 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21937 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.747 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237070+0000", 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237069+0000", 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.748 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237071+0000", 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.085e-06 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237074+0000", 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.225e-06 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237076+0000", 2024-09-13T05:03:49.749 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2869999999999999e-06 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274916+0000", 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037839823000000002 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.274993+0000", 2024-09-13T05:03:49.750 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6678e-05 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275119+0000", 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00012625900000000001 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21980 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 24576~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.487323257, 2024-09-13T05:03:49.751 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038162809999999998, 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21980 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.752 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237078+0000", 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237081+0000", 2024-09-13T05:03:49.753 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.084e-06 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237084+0000", 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.903e-06 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237086+0000", 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2929999999999999e-06 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.754 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275148+0000", 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038061876000000001 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275189+0000", 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0844999999999999e-05 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275241+0000", 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2014e-05 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.755 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21981 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 122880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.487313236000006, 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038309642999999997, 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.756 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21981 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.757 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237088+0000", 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237089+0000", 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1310000000000001e-06 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237092+0000", 2024-09-13T05:03:49.758 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8590000000000001e-06 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237094+0000", 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3250000000000002e-06 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275273+0000", 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038179307000000003 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.759 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275319+0000", 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5603999999999999e-05 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275398+0000", 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.8625e-05 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21982 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 221184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.760 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.487297513000001, 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038512884999999997, 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21982 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.761 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237104+0000", 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237103+0000", 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.762 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237105+0000", 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.7069999999999999e-06 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237108+0000", 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4379999999999999e-06 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.237111+0000", 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5830000000000001e-06 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.763 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275438+0000", 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038327410999999999 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275520+0000", 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.1118000000000005e-05 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.275617+0000", 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.7077000000000006e-05 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.764 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21990 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1007616~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.485395343999997, 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037782373000000001, 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.765 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21990 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.766 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239006+0000", 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239007+0000", 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.0750000000000001e-06 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239010+0000", 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7949999999999999e-06 2024-09-13T05:03:49.767 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239012+0000", 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3539999999999998e-06 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276701+0000", 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037688746000000002 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.768 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276736+0000", 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5618999999999999e-05 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276788+0000", 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2008000000000001e-05 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21991 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1138688~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:49.769 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.485387384999996, 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037958160999999997, 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21991 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:49.770 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239014+0000", 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.771 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239015+0000", 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.125e-06 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239018+0000", 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.74e-06 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239020+0000", 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.772 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276813+0000", 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037793491999999998 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276880+0000", 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 6.6484999999999998e-05 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.276972+0000", 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.2410999999999993e-05 2024-09-13T05:03:49.773 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21992 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1236992~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.485379696999999, 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038083822000000003, 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21992 2024-09-13T05:03:49.774 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239022+0000", 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.775 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239021+0000", 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239025+0000", 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.6770000000000001e-06 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239028+0000", 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0299999999999998e-06 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.776 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239030+0000", 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277006+0000", 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037975209000000003 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277049+0000", 2024-09-13T05:03:49.777 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3269000000000002e-05 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277105+0000", 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.6610000000000002e-05 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21993 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1368064~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:49.778 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.485369004999995, 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038200425000000003, 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21993 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239032+0000", 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.780 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239033+0000", 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.1260000000000001e-06 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239036+0000", 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.807e-06 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239039+0000", 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.5670000000000002e-06 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.781 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277136+0000", 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038097909999999999 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277182+0000", 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.5314e-05 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277233+0000", 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.0930999999999997e-05 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.782 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21994 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1466368~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.485360764000006, 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038316694999999998, 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.783 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21994 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.784 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239040+0000", 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239041+0000", 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.119e-06 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239044+0000", 2024-09-13T05:03:49.785 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7520000000000001e-06 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239046+0000", 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.295e-06 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277257+0000", 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038210433000000002 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.786 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277305+0000", 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8075000000000003e-05 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277357+0000", 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2250000000000003e-05 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:21995 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 1564672~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.787 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.485337103000006, 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038410061000000002, 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 21995 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.788 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239064+0000", 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.789 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239065+0000", 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.646e-06 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239073+0000", 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 7.6469999999999998e-06 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.239076+0000", 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.655e-06 2024-09-13T05:03:49.790 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277381+0000", 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038305114000000001 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277421+0000", 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0472999999999997e-05 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.277474+0000", 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.2935000000000001e-05 2024-09-13T05:03:49.791 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22005 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2482176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.483694643999996, 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037872712000000003, 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.792 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22005 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240707+0000", 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.793 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240706+0000", 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240708+0000", 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.144e-06 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.794 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240710+0000", 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6910000000000002e-06 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240713+0000", 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.2239999999999998e-06 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278490+0000", 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037777273 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.795 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278530+0000", 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.0102999999999999e-05 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278579+0000", 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.9487000000000002e-05 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22006 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2580480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.796 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.483686410999994, 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037971416000000001, 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22006 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.797 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240715+0000", 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.798 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240716+0000", 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 9.8100000000000001e-07 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240718+0000", 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7300000000000001e-06 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240721+0000", 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7480000000000002e-06 2024-09-13T05:03:49.799 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278601+0000", 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037879968999999999 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278639+0000", 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7548e-05 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278686+0000", 2024-09-13T05:03:49.800 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7651000000000002e-05 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22007 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2711552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.483463682000007, 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037858791000000003, 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.801 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22007 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240938+0000", 2024-09-13T05:03:49.802 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240723+0000", 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9997854 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240939+0000", 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00021630600000000001 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.803 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240944+0000", 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.8450000000000002e-06 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240947+0000", 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.0970000000000002e-06 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278705+0000", 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037758134999999998 2024-09-13T05:03:49.804 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278744+0000", 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.9069999999999997e-05 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278796+0000", 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.1949e-05 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.805 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22008 3.d 3:b54e85d7:::rbd_data.117c7aa26b99.000000000000003a:d [read 2809856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.483451133000003, 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037951249999999999, 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "started", 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22008 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.806 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240950+0000", 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:49.807 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240951+0000", 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.2160000000000001e-06 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240954+0000", 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8389999999999998e-06 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.240956+0000", 2024-09-13T05:03:49.808 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.3580000000000001e-06 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278817+0000", 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.037860540999999998 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278854+0000", 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.7348999999999998e-05 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.809 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.278901+0000", 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.7193e-05 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22014 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.481411906999995, 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.038928332000000003, 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:49.810 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22014 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.811 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242989+0000", 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242991+0000", 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.5489999999999999e-06 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.812 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242994+0000", 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.9069999999999999e-06 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242997+0000", 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4189999999999998e-06 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.813 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279484+0000", 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036486672999999997 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279528+0000", 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.4502999999999997e-05 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279729+0000", 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00020125500000000001 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.814 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281831+0000", 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0021010669999999999 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281839+0000", 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 8.4260000000000003e-06 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281883+0000", 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4.3720000000000002e-05 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.815 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281918+0000", 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.5049999999999998e-05 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "description": "osd_op(client.4476.0:22015 3.d 3:b3e38454:::rbd_object_map.117cacca117a:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "initiated_at": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "age": 85.481402127999999, 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.039017450000000002, 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "type_data": { 2024-09-13T05:03:49.816 INFO:teuthology.orchestra.run.smithi110.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_info": { 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client": "client.4476", 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "tid": 22015 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "events": [ 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "initiated", 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.817 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "throttled", 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "header_read", 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.242999+0000", 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 4294967296 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "all_read", 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243000+0000", 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 1.415e-06 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.818 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "dispatched", 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243003+0000", 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.8430000000000001e-06 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "queued_for_pg", 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.243006+0000", 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.6709999999999999e-06 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "reached_pg", 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.279996+0000", 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.036990442999999998 2024-09-13T05:03:49.819 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "started", 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280054+0000", 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 5.7608000000000001e-05 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "waiting for subops from 5", 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.280247+0000", 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.00019299399999999999 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "op_commit", 2024-09-13T05:03:49.820 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281930+0000", 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 0.0016834389999999999 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281955+0000", 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.4666999999999999e-05 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "commit_sent", 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.281983+0000", 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 2.7540000000000001e-05 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: { 2024-09-13T05:03:49.821 INFO:teuthology.orchestra.run.smithi110.stdout: "event": "done", 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: "time": "2024-09-13T05:02:24.282017+0000", 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: "duration": 3.4037e-05 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-09-13T05:03:49.822 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-09-13T05:03:49.823 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:49.823+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:49.824 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-13T05:03:49.924 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:49.924+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:49.945 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:49.945 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:49.945 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:49.945 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:49.955 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-13T05:03:49.980 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:03:50.039 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:50.039+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:50.108 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:50.109 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:50.109 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:50.109 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:50.109 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:50.120 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-13T05:03:50.126 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:50.125+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:50.227 INFO:tasks.ceph.osd.5.smithi110.stderr:2024-09-13T05:03:50.226+0000 7fdb644fd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33004) UID: 0 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39877 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.268 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611138522000005, 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301850000000001, 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39877 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:50.269 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655489+0000", 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.270 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655491+0000", 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3039999999999999e-06 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655497+0000", 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8599999999999998e-06 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655501+0000", 2024-09-13T05:03:50.271 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4989999999999996e-06 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688783+0000", 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033282113000000002 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688791+0000", 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4370000000000001e-06 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.272 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39879 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611122524999999, 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318624999999998, 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.273 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39879 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.274 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655505+0000", 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655507+0000", 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.136e-06 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655512+0000", 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6559999999999997e-06 2024-09-13T05:03:50.275 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655517+0000", 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7350000000000004e-06 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688819+0000", 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301411000000003 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.276 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688824+0000", 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0080000000000002e-06 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39880 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611105729000002, 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033334837999999999, 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.277 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39880 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655522+0000", 2024-09-13T05:03:50.278 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655521+0000", 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655524+0000", 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4329999999999998e-06 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.279 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655530+0000", 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.246e-06 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655535+0000", 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7049999999999998e-06 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688846+0000", 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033311689999999998 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.280 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688857+0000", 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0139000000000001e-05 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.281 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39882 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611088756000001, 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369012000000003, 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39882 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.282 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655539+0000", 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.283 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655538+0000", 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655540+0000", 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0820000000000001e-06 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655546+0000", 2024-09-13T05:03:50.284 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4589999999999997e-06 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655551+0000", 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8189999999999999e-06 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688898+0000", 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033347112999999998 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.285 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688908+0000", 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8889999999999997e-06 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.286 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39884 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611072362000002, 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033369261999999997, 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39884 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.287 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.288 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655555+0000", 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655557+0000", 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.261e-06 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655563+0000", 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9270000000000002e-06 2024-09-13T05:03:50.289 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655568+0000", 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7029999999999997e-06 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688919+0000", 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033351571000000003 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.290 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688924+0000", 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1710000000000001e-06 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39885 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611055325999999, 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033380252999999999, 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.291 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39885 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.292 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655572+0000", 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655574+0000", 2024-09-13T05:03:50.293 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2879999999999999e-06 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655580+0000", 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0349999999999998e-06 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655599+0000", 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9293999999999999e-05 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.294 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688946+0000", 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033346801000000002 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688952+0000", 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2040000000000002e-06 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.295 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39887 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611021742999995, 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033360946000000002, 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39887 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.296 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655606+0000", 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.297 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655605+0000", 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655608+0000", 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8499999999999998e-06 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655615+0000", 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2019999999999998e-06 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.298 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655620+0000", 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0699999999999997e-06 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688963+0000", 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033342179999999999 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.299 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688967+0000", 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1239999999999998e-06 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.300 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39891 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.611002614, 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033375100999999997, 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39891 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.301 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655625+0000", 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.302 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655624+0000", 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655631+0000", 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8650000000000003e-06 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655638+0000", 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3980000000000004e-06 2024-09-13T05:03:50.303 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655643+0000", 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0849999999999996e-06 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.688991+0000", 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033348430999999998 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.304 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689000+0000", 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7390000000000005e-06 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39893 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610924745000005, 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033310943000000003, 2024-09-13T05:03:50.305 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39893 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:50.306 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655703+0000", 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655702+0000", 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.307 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655706+0000", 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.715e-06 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655715+0000", 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9609999999999994e-06 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.308 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655721+0000", 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0260000000000004e-06 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689010+0000", 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033288813 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689014+0000", 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0520000000000003e-06 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.309 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39898 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610864000000007, 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033307188000000001, 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39898 2024-09-13T05:03:50.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655764+0000", 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655763+0000", 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655766+0000", 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.605e-06 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655772+0000", 2024-09-13T05:03:50.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3990000000000001e-06 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655796+0000", 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3966999999999999e-05 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689063+0000", 2024-09-13T05:03:50.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033266962999999997 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689071+0000", 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.644e-06 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39901 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610803786000005, 2024-09-13T05:03:50.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033306648000000001, 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39901 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655824+0000", 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655823+0000", 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655826+0000", 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4499999999999998e-06 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655832+0000", 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3010000000000003e-06 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655837+0000", 2024-09-13T05:03:50.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7450000000000003e-06 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689124+0000", 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033287678000000001 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689130+0000", 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9390000000000002e-06 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39903 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610786458999996, 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033327681999999997, 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.319 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39903 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655841+0000", 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655843+0000", 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2520000000000002e-06 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655849+0000", 2024-09-13T05:03:50.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.2990000000000002e-06 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655854+0000", 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0610000000000003e-06 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689160+0000", 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305609 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689169+0000", 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9029999999999993e-06 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39907 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610757286999998, 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323461999999998, 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.323 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39907 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655870+0000", 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655873+0000", 2024-09-13T05:03:50.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0639999999999998e-06 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655880+0000", 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0430000000000002e-06 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655885+0000", 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3940000000000004e-06 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689187+0000", 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301431999999999 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689194+0000", 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0489999999999998e-06 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39909 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:50.327 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610737603000004, 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033318226999999999, 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39909 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:50.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655890+0000", 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655889+0000", 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.329 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655892+0000", 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4109999999999999e-06 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655898+0000", 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5579999999999997e-06 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655903+0000", 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6269999999999999e-06 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689204+0000", 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033301114999999999 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689208+0000", 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.002e-06 2024-09-13T05:03:50.331 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39911 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610720169999993, 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033323441000000002, 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.332 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39911 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655907+0000", 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655909+0000", 2024-09-13T05:03:50.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3479999999999998e-06 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655916+0000", 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655921+0000", 2024-09-13T05:03:50.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689226+0000", 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033305221000000003 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689231+0000", 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8230000000000002e-06 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.336 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39912 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610702528000004, 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033328506000000001, 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.337 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39912 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655925+0000", 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655927+0000", 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1880000000000001e-06 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655933+0000", 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1410000000000002e-06 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655953+0000", 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0516000000000001e-05 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689249+0000", 2024-09-13T05:03:50.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033295103 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689253+0000", 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9570000000000001e-06 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39914 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610668352000005, 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315965000000003, 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39914 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:50.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655959+0000", 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655966+0000", 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3549999999999999e-06 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655974+0000", 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7079999999999997e-06 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655980+0000", 2024-09-13T05:03:50.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8089999999999998e-06 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689270+0000", 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033290893000000002 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689275+0000", 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6709999999999998e-06 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.345 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39915 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610643002000003, 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033312201, 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39915 2024-09-13T05:03:50.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655985+0000", 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655984+0000", 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655986+0000", 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.424e-06 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655993+0000", 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5629999999999997e-06 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.655998+0000", 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8690000000000003e-06 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689292+0000", 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033293705999999999 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689297+0000", 2024-09-13T05:03:50.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0950000000000003e-06 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39916 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610625329000001, 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033315248999999998, 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.350 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39916 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656002+0000", 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656006+0000", 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2780000000000004e-06 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656013+0000", 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5379999999999999e-06 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656022+0000", 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0440000000000001e-06 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689313+0000", 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033291447000000002 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689317+0000", 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3429999999999998e-06 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:39918 3.12 3.e9db0652 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 78.610601125000002, 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033319033999999997, 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.354 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "reached pg", 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 39918 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656026+0000", 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656028+0000", 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2960000000000001e-06 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656035+0000", 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4150000000000004e-06 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.656040+0000", 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9779999999999996e-06 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689337+0000", 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.033297098999999997 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.689345+0000", 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6060000000000003e-06 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.358 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:50.360 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-13T05:03:50.360 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:50.359+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:50.460 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:50.460+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:50.507 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:50.508 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:50.508 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:50.508 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:50.516 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-13T05:03:50.561 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:50.561+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:50.625 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:50.625 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:50.625 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:50.631 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-13T05:03:50.676 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:50.675+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:50.739 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:50.739 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47915 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 679936~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375873053999996, 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019846084999999999, 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.740 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47915 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.741 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.742 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362087+0000", 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362094+0000", 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3150000000000003e-06 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362101+0000", 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6660000000000002e-06 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.743 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.744 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.744 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362106+0000", 2024-09-13T05:03:50.744 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0969999999999996e-06 2024-09-13T05:03:50.744 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.744 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381745+0000", 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019639012000000001 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381834+0000", 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9024999999999996e-05 2024-09-13T05:03:50.745 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381933+0000", 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9334999999999997e-05 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.746 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47916 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 737280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.747 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:50.747 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375850010999997, 2024-09-13T05:03:50.747 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020037577000000001, 2024-09-13T05:03:50.747 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.747 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.747 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.748 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.748 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47916 2024-09-13T05:03:50.748 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.748 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.748 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.749 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.750 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.750 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362110+0000", 2024-09-13T05:03:50.750 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.750 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.750 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362112+0000", 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.323e-06 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362119+0000", 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.5880000000000003e-06 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.751 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362123+0000", 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4900000000000002e-06 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.381961+0000", 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019838214 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.752 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382042+0000", 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0587000000000006e-05 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382148+0000", 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000105738 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.753 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.754 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.754 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.754 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47917 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 835584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.754 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:50.754 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375832678999998, 2024-09-13T05:03:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020247431, 2024-09-13T05:03:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.755 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47917 2024-09-13T05:03:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.756 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.757 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362128+0000", 2024-09-13T05:03:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.758 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362127+0000", 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362129+0000", 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2840000000000001e-06 2024-09-13T05:03:50.759 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362135+0000", 2024-09-13T05:03:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0589999999999999e-06 2024-09-13T05:03:50.760 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362140+0000", 2024-09-13T05:03:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6999999999999999e-06 2024-09-13T05:03:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.761 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382184+0000", 2024-09-13T05:03:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020044217999999999 2024-09-13T05:03:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.762 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382274+0000", 2024-09-13T05:03:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9436000000000002e-05 2024-09-13T05:03:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.763 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382375+0000", 2024-09-13T05:03:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101099 2024-09-13T05:03:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.764 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47918 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 892928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375816134999994, 2024-09-13T05:03:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020434145000000001, 2024-09-13T05:03:50.765 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47918 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.766 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.767 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.768 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362144+0000", 2024-09-13T05:03:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.769 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362146+0000", 2024-09-13T05:03:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4310000000000001e-06 2024-09-13T05:03:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.770 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362152+0000", 2024-09-13T05:03:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.173e-06 2024-09-13T05:03:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.771 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362157+0000", 2024-09-13T05:03:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7589999999999997e-06 2024-09-13T05:03:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382403+0000", 2024-09-13T05:03:50.772 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020245737 2024-09-13T05:03:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382485+0000", 2024-09-13T05:03:50.773 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.1766000000000003e-05 2024-09-13T05:03:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382578+0000", 2024-09-13T05:03:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3639000000000003e-05 2024-09-13T05:03:50.774 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47919 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 933888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.775 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375799009999994, 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020633708000000001, 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47919 2024-09-13T05:03:50.776 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.777 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362161+0000", 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.778 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362163+0000", 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1399999999999998e-06 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362169+0000", 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7810000000000002e-06 2024-09-13T05:03:50.779 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362173+0000", 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7199999999999997e-06 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382607+0000", 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020433705999999999 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.780 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382694+0000", 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6403000000000006e-05 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382795+0000", 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010132999999999999 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.781 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47920 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 991232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375782782000002, 2024-09-13T05:03:50.782 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020858714, 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47920 2024-09-13T05:03:50.783 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.784 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.785 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362177+0000", 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.786 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362180+0000", 2024-09-13T05:03:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4439999999999998e-06 2024-09-13T05:03:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362186+0000", 2024-09-13T05:03:50.787 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0440000000000001e-06 2024-09-13T05:03:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362190+0000", 2024-09-13T05:03:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5299999999999998e-06 2024-09-13T05:03:50.788 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382839+0000", 2024-09-13T05:03:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020648561999999999 2024-09-13T05:03:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.789 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.382936+0000", 2024-09-13T05:03:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7625000000000001e-05 2024-09-13T05:03:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.790 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383036+0000", 2024-09-13T05:03:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9840999999999995e-05 2024-09-13T05:03:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.791 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47921 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1089536~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375765971000007, 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021078105999999999, 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.792 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47921 2024-09-13T05:03:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.793 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.794 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.795 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362194+0000", 2024-09-13T05:03:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.796 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362196+0000", 2024-09-13T05:03:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0940000000000002e-06 2024-09-13T05:03:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362202+0000", 2024-09-13T05:03:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9499999999999998e-06 2024-09-13T05:03:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362207+0000", 2024-09-13T05:03:50.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6949999999999999e-06 2024-09-13T05:03:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383083+0000", 2024-09-13T05:03:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020876530000000001 2024-09-13T05:03:50.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383177+0000", 2024-09-13T05:03:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.3566000000000005e-05 2024-09-13T05:03:50.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383272+0000", 2024-09-13T05:03:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5600000000000006e-05 2024-09-13T05:03:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.801 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47922 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1146880~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:50.802 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375749353000003, 2024-09-13T05:03:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021270397999999999, 2024-09-13T05:03:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47922 2024-09-13T05:03:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362211+0000", 2024-09-13T05:03:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362210+0000", 2024-09-13T05:03:50.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362213+0000", 2024-09-13T05:03:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-06 2024-09-13T05:03:50.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362218+0000", 2024-09-13T05:03:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8640000000000001e-06 2024-09-13T05:03:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362223+0000", 2024-09-13T05:03:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8280000000000002e-06 2024-09-13T05:03:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383316+0000", 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021092847000000001 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383397+0000", 2024-09-13T05:03:50.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0883999999999993e-05 2024-09-13T05:03:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383481+0000", 2024-09-13T05:03:50.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4248999999999999e-05 2024-09-13T05:03:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.812 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47923 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1187840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375732775000003, 2024-09-13T05:03:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021484315, 2024-09-13T05:03:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.813 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47923 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362227+0000", 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362229+0000", 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1469999999999999e-06 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362235+0000", 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9240000000000004e-06 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362240+0000", 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8919999999999999e-06 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383522+0000", 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021281608 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383614+0000", 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.2193000000000003e-05 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383712+0000", 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8009000000000003e-05 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47924 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1245184~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375696421000001, 2024-09-13T05:03:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021630257999999999, 2024-09-13T05:03:50.820 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47924 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.821 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362264+0000", 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362263+0000", 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362266+0000", 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8449999999999999e-06 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362273+0000", 2024-09-13T05:03:50.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4269999999999996e-06 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362278+0000", 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.395e-06 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383757+0000", 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021478524999999998 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.824 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383835+0000", 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8404000000000004e-05 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383894+0000", 2024-09-13T05:03:50.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9043999999999999e-05 2024-09-13T05:03:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.826 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47925 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1302528~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375677476999996, 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021776837, 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.827 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47925 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362283+0000", 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362282+0000", 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362285+0000", 2024-09-13T05:03:50.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3530000000000002e-06 2024-09-13T05:03:50.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362290+0000", 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6860000000000003e-06 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362295+0000", 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:50.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383928+0000", 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021632707000000001 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.383982+0000", 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4503999999999999e-05 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384060+0000", 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7191e-05 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.835 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47926 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1343488~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375660843000006, 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021964198000000001, 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.836 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47926 2024-09-13T05:03:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362299+0000", 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362305+0000", 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1859999999999997e-06 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362314+0000", 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7329999999999992e-06 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362319+0000", 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1510000000000003e-06 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384084+0000", 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021764761000000001 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384167+0000", 2024-09-13T05:03:50.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2727999999999998e-05 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384264+0000", 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7009000000000006e-05 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47927 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1400832~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375634841999997, 2024-09-13T05:03:50.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022186992999999999, 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47927 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362325+0000", 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362339+0000", 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4405e-05 2024-09-13T05:03:50.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362347+0000", 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5379999999999997e-06 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362352+0000", 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.305e-06 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384310+0000", 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.021958083999999999 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384406+0000", 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5957000000000003e-05 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384512+0000", 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000106059 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.845 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47928 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1441792~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375602903000001, 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022370134, 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47928 2024-09-13T05:03:50.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362357+0000", 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362359+0000", 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.356e-06 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362365+0000", 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.913e-06 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362371+0000", 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9950000000000002e-06 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384544+0000", 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02217293 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384628+0000", 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3710999999999999e-05 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384727+0000", 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9649999999999996e-05 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47929 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1499136~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375584115999999, 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022513951000000001, 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.850 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47929 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362376+0000", 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362378+0000", 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0990000000000001e-06 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362384+0000", 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7939999999999999e-06 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362388+0000", 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.7430000000000002e-06 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384754+0000", 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022365290999999999 2024-09-13T05:03:50.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384807+0000", 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3841000000000002e-05 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384890+0000", 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2565999999999995e-05 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.854 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47930 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375567996000001, 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022623937, 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47930 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.855 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362392+0000", 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.856 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362394+0000", 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0779999999999998e-06 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362399+0000", 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4960000000000004e-06 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362408+0000", 2024-09-13T05:03:50.857 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4149999999999999e-06 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384916+0000", 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022508542999999999 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.384965+0000", 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8798999999999998e-05 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385016+0000", 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0923e-05 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47931 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1597440~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375545935000005, 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022801735, 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.859 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47931 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.860 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362414+0000", 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362416+0000", 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9759999999999998e-06 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362422+0000", 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6319999999999996e-06 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362426+0000", 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5789999999999997e-06 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385052+0000", 2024-09-13T05:03:50.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022625915999999999 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385135+0000", 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3002000000000003e-05 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385216+0000", 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0939000000000006e-05 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:47932 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.375528970000005, 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022973304999999999, 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 47932 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.864 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362431+0000", 2024-09-13T05:03:50.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362433+0000", 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3520000000000001e-06 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362439+0000", 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1709999999999999e-06 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.362444+0000", 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5050000000000001e-06 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385245+0000", 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.02280161 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385322+0000", 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6574000000000002e-05 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.385405+0000", 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2470000000000002e-05 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50258 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3178496~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.063494571000007, 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019968091, 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.868 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50258 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674466+0000", 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674465+0000", 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674476+0000", 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0168e-05 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674479+0000", 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6009999999999999e-06 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674482+0000", 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6599999999999999e-06 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694219+0000", 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019737607000000001 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694321+0000", 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000101136 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694434+0000", 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011312800000000001 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:50259 3.0 3:042ae13f:::rbd_data.117c6173c20e.0000000000000001:14 [read 3235840~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-09-13T05:03:50.872 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.063475463000003, 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.020102894999999999, 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 50259 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:50.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674485+0000", 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674484+0000", 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:50.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674486+0000", 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3030000000000001e-06 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674489+0000", 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8459999999999999e-06 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.674491+0000", 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3939999999999999e-06 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694459+0000", 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.019967849999999999 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694521+0000", 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.1733e-05 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.694588+0000", 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7013000000000004e-05 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.876 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.877 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:50.877 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:50.877 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:50.878 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-13T05:03:50.879 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:50.878+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:50.965 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:03:50.979 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:50.978+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:51.035 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:51.035 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:51.036 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:51.036 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:51.044 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-13T05:03:51.090 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:51.089+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:51.152 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:51.152 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:51.152 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:51.161 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-13T05:03:51.201 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:51.201+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:51.281 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:51.280+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:51.309 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.309 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:32406 3.9 e35/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:head v 35'160, mlcod=35'158)", 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 83.504268842000002, 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0034244140000000002, 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.310 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803448+0000", 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.803446+0000", 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999981 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805187+0000", 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0017404149999999999 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.311 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805204+0000", 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7076999999999999e-05 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805210+0000", 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.55e-06 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805258+0000", 2024-09-13T05:03:51.312 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8164999999999997e-05 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.805277+0000", 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8227999999999999e-05 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806826+0000", 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0015497849999999999 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.313 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:27.806872+0000", 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5973999999999997e-05 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:40346 3.4 e37/19 3:2296c5ff:::rbd_data.117c4a06f7bc.000000000000022a:head v 37'211, mlcod=37'208)", 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 79.363199808000005, 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033675749999999998, 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.314 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944517+0000", 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.315 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944516+0000", 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944535+0000", 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9528e-05 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944557+0000", 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1733999999999999e-05 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.316 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944566+0000", 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6900000000000004e-06 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944617+0000", 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0204999999999998e-05 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.944639+0000", 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2031e-05 2024-09-13T05:03:51.317 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947843+0000", 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032045630000000001 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:31.947885+0000", 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.1304999999999998e-05 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.318 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:48647 3.4 e41/19 3:221c6fcb:::rbd_data.117cbeb44a94.0000000000000002:head v 41'291, mlcod=41'289)", 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 73.728847014999999, 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.004488438, 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.319 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578870+0000", 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.578869+0000", 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580522+0000", 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016534550000000001 2024-09-13T05:03:51.320 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580546+0000", 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4060999999999999e-05 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580557+0000", 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.135e-05 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580612+0000", 2024-09-13T05:03:51.321 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4252999999999997e-05 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.580642+0000", 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.0386999999999999e-05 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583331+0000", 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026889399999999999 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.322 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:37.583358+0000", 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7242000000000001e-05 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52984 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'397, mlcod=44'393)", 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.949732940999994, 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002761707, 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.323 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357984+0000", 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357983+0000", 2024-09-13T05:03:51.324 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.357989+0000", 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.285e-06 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358002+0000", 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2379e-05 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.325 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358009+0000", 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7859999999999999e-06 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358049+0000", 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0571999999999997e-05 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.358068+0000", 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8427000000000002e-05 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.326 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360708+0000", 2024-09-13T05:03:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026407560000000002 2024-09-13T05:03:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.327 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360746+0000", 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7212000000000002e-05 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:52985 3.4 e44/19 3:2352d7ef:::rbd_object_map.117c12c41b07:head v 44'398, mlcod=44'395)", 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:51.328 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 70.948445938999996, 2024-09-13T05:03:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0023931849999999999, 2024-09-13T05:03:51.329 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359271+0000", 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.330 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359270+0000", 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359278+0000", 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.5070000000000003e-06 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.331 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359292+0000", 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4537e-05 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.359300+0000", 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7780000000000007e-06 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360284+0000", 2024-09-13T05:03:51.332 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098367500000000004 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.360304+0000", 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0194999999999999e-05 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.333 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361631+0000", 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013273810000000001 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:40.361664+0000", 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2818999999999999e-05 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(client.4476.0:83276 3.4 e60/19 3:26b302fe:::rbd_data.117c763fc6aa.0000000000000002:head v 60'726, mlcod=59'724)", 2024-09-13T05:03:51.334 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 50.332176064000002, 2024-09-13T05:03:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0039477289999999996, 2024-09-13T05:03:51.335 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975541+0000", 2024-09-13T05:03:51.336 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.975540+0000", 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976050+0000", 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050985799999999997 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.337 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976069+0000", 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8393000000000002e-05 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976098+0000", 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.9386e-05 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.338 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976193+0000", 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4728999999999994e-05 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.976217+0000", 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3794e-05 2024-09-13T05:03:51.339 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979453+0000", 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032362860000000001 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:00.979488+0000", 2024-09-13T05:03:51.340 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5629000000000001e-05 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98341 3.f 3:f15ef2ff:::rbd_data.117cf03415a9.00000000000000bb:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.891706280999998, 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021618190000000002, 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.341 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98341 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416011+0000", 2024-09-13T05:03:51.342 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416009+0000", 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416015+0000", 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8429999999999998e-06 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.343 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416028+0000", 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2198000000000001e-05 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416035+0000", 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1409999999999999e-06 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416207+0000", 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00017254999999999999 2024-09-13T05:03:51.344 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416310+0000", 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000103286 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416371+0000", 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0238000000000002e-05 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.416872+0000", 2024-09-13T05:03:51.345 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00050110900000000002 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418099+0000", 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001227271 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418141+0000", 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.2089999999999999e-05 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.346 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.418172+0000", 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1137999999999999e-05 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:98738 3.f 3:f4eda464:::rbd_data.117cf03415a9.000000000000007e:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.479951636000003, 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.018816306000000001, 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.347 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 98738 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.348 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827765+0000", 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827764+0000", 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.349 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827772+0000", 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.585e-06 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827793+0000", 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.075e-05 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827800+0000", 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8619999999999997e-06 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.350 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827849+0000", 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9579000000000001e-05 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.827948+0000", 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8382999999999996e-05 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.831301+0000", 2024-09-13T05:03:51.351 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.003353698 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846517+0000", 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.015215975 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846526+0000", 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2930000000000001e-06 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.352 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846557+0000", 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1012000000000001e-05 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846581+0000", 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.4803999999999998e-05 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.353 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop_reply(client.4476.0:98738 3.f e66/55 ondisk, result = 0)", 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 41.466699853000001, 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0055032170000000004, 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.354 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841017+0000", 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841016+0000", 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841022+0000", 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.711e-06 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.355 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841038+0000", 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6419999999999999e-05 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.841050+0000", 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.185e-05 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846506+0000", 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.005456079 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.356 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846513+0000", 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.7510000000000002e-06 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:09.846520+0000", 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.34e-06 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.357 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.4.0:2845 3.9 e79/19 3:9799fd9b:::rbd_data.117c394b9965.000000000000001e:1f v 79'412, mlcod=77'407)", 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.324103666999999, 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002201724, 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.358 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983613+0000", 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983612+0000", 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983621+0000", 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7639999999999994e-06 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.359 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983642+0000", 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1631e-05 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.983649+0000", 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.6139999999999997e-06 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.984376+0000", 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00072730199999999998 2024-09-13T05:03:51.360 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.984393+0000", 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6738000000000001e-05 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985760+0000", 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0013673369999999999 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:22.985815+0000", 2024-09-13T05:03:51.361 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4478000000000003e-05 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102158 3.f 3:f4db7252:::rbd_data.117c394b9965.0000000000000010:head [delete] snapc 1f=[] ondisk+write+known_if_redirected+supports_pool_eio e79)", 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 28.250572977000001, 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002237228, 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.362 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102158 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.363 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057144+0000", 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057141+0000", 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057155+0000", 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3583e-05 2024-09-13T05:03:51.364 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057176+0000", 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0815999999999999e-05 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057187+0000", 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0841e-05 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057255+0000", 2024-09-13T05:03:51.365 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8226999999999996e-05 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057343+0000", 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8294999999999994e-05 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.057459+0000", 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000115439 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.366 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.058518+0000", 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.001059286 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059231+0000", 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071358399999999996 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059301+0000", 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.9479000000000003e-05 2024-09-13T05:03:51.367 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:23.059381+0000", 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.0153000000000003e-05 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1861 3.4 e83/19 3:249fe31d:::rbd_data.117cc3f2292.0000000000000064:1b v 83'980, mlcod=77'977)", 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.306232328, 2024-09-13T05:03:51.368 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022663900000000001, 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.369 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001484+0000", 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001490+0000", 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7749999999999998e-06 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.370 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001497+0000", 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3069999999999997e-06 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001504+0000", 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2819999999999999e-06 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001592+0000", 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8015000000000003e-05 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.371 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.001614+0000", 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1163e-05 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.003705+0000", 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002091337 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.003751+0000", 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5890999999999998e-05 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.372 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102933 3.f 3:f3a9a226:::rbd_data.117cc3f2292.000000000000000a:head [delete] snapc 1b=[] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 24.239291142999999, 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002252291, 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.373 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102933 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068426+0000", 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.374 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068423+0000", 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068433+0000", 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.8959999999999998e-06 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068454+0000", 2024-09-13T05:03:51.375 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0372000000000001e-05 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068464+0000", 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0828e-05 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068528+0000", 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3744999999999994e-05 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.376 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068625+0000", 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6457000000000002e-05 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.068735+0000", 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00011031600000000001 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.069588+0000", 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00085322700000000002 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.377 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070578+0000", 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00098980800000000001 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070634+0000", 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6036999999999999e-05 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.070678+0000", 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3914e-05 2024-09-13T05:03:51.378 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:102997 3.f 3:f4216607:::rbd_header.117c7beb7c1e:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 23.308534384000001, 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021657629999999998, 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.379 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 102997 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999182+0000", 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.380 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999180+0000", 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999976 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999193+0000", 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2986999999999999e-05 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999209+0000", 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5474000000000001e-05 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.381 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999219+0000", 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0745e-05 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999288+0000", 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8761000000000001e-05 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999449+0000", 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000160435 2024-09-13T05:03:51.382 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 4", 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:27.999958+0000", 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000509846 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.000669+0000", 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00071006499999999996 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.383 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001250+0000", 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.000581067 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001285+0000", 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4993000000000002e-05 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:28.001348+0000", 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3600000000000001e-05 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.384 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1956 3.4 e92/19 3:2031bc64:::rbd_data.117c4a06f7bc.0000000000000096:12 v 92'1040, mlcod=91'1037)", 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:35.918410+0000", 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.389306787000001, 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0022804019999999999, 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918410+0000", 2024-09-13T05:03:51.385 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918410+0000", 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918409+0000", 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.386 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918416+0000", 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.0949999999999998e-06 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918425+0000", 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3650000000000004e-06 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.387 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918432+0000", 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3409999999999997e-06 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918517+0000", 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5452999999999997e-05 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.918537+0000", 2024-09-13T05:03:51.388 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9862999999999999e-05 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.920657+0000", 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0021194640000000002 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.920690+0000", 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3643000000000003e-05 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.389 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:1960 3.4 e92/19 3:234d8752:::rbd_data.117c4a06f7bc.000000000000001a:12 v 92'1048, mlcod=92'1044)", 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:35.928645+0000", 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 15.379072157, 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024360509999999998, 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928645+0000", 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.390 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928645+0000", 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928643+0000", 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.391 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928653+0000", 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.6909999999999992e-06 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928667+0000", 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3645000000000001e-05 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.928674+0000", 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1099999999999997e-06 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.392 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.929573+0000", 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00089939600000000001 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.929596+0000", 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3033000000000001e-05 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.931042+0000", 2024-09-13T05:03:51.393 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0014453000000000001 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:35.931081+0000", 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9067999999999999e-05 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.4.0:3144 3.9 e94/19 3:904e1775:::rbd_data.117c9ee8ea0.000000000000001c:10 v 94'434, mlcod=92'430)", 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:37.924588+0000", 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 13.383128510000001, 2024-09-13T05:03:51.394 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0033994369999999999, 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924588+0000", 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924588+0000", 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.395 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924587+0000", 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924596+0000", 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4970000000000001e-06 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924612+0000", 2024-09-13T05:03:51.396 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.5781999999999999e-05 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924619+0000", 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7740000000000004e-06 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924670+0000", 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0547999999999998e-05 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.397 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.924692+0000", 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2507999999999999e-05 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.927948+0000", 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0032559389999999998 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:37.927988+0000", 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.9375999999999997e-05 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.398 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:2020 3.4 e101/19 3:2034f8e0:::rbd_data.117c6c62fa22.00000000000000a0:9 v 101'1079, mlcod=98'1076)", 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:44.903114+0000", 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.404602616, 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0028115060000000001, 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.399 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903114+0000", 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903114+0000", 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903113+0000", 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:51.400 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903124+0000", 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.1564e-05 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903144+0000", 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.9545e-05 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.401 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903152+0000", 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.7560000000000005e-06 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903239+0000", 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6723000000000003e-05 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903258+0000", 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.8538000000000001e-05 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.402 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.905893+0000", 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0026348589999999998 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.905926+0000", 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.3149999999999999e-05 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.403 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:2021 3.0 e101/19 3:03ac79a9:::rbd_data.117c6c62fa22.0000000000000021:9 v 101'582, mlcod=101'578)", 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:44.903166+0000", 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.404550951, 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025739629999999999, 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903166+0000", 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.404 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903166+0000", 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903166+0000", 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903174+0000", 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.9489999999999996e-06 2024-09-13T05:03:51.405 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903179+0000", 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6359999999999999e-06 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903183+0000", 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8809999999999998e-06 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903283+0000", 2024-09-13T05:03:51.406 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.9796000000000005e-05 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.903305+0000", 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1962000000000001e-05 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.905705+0000", 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0024000330000000002 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.407 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.905740+0000", 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5002000000000001e-05 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_repop(osd.1.0:2022 3.0 e101/19 3:03668f61:::rbd_data.117c6c62fa22.000000000000008d:9 v 101'584, mlcod=101'578)", 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:03:44.904018+0000", 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 6.4036988109999999, 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0035546969999999999, 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.408 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904018+0000", 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904018+0000", 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.409 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904017+0000", 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.999999 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904024+0000", 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.63e-06 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904034+0000", 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.0188e-05 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.410 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904040+0000", 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.421e-06 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904925+0000", 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00088427500000000001 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.904946+0000", 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.1155e-05 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.411 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.907537+0000", 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0025916929999999999 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:03:44.907573+0000", 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5188000000000003e-05 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.412 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:51.414 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-13T05:03:51.415 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:51.413+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:51.514 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:51.514+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:51.561 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:51.561 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:51.561 INFO:teuthology.orchestra.run.smithi053.stdout: "num_ops": 0 2024-09-13T05:03:51.561 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:51.570 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-13T05:03:51.573 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-13T05:03:51.615 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:51.615+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:51.678 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:51.678 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [], 2024-09-13T05:03:51.678 INFO:teuthology.orchestra.run.smithi053.stdout: "complaint_time": 30, 2024-09-13T05:03:51.678 INFO:teuthology.orchestra.run.smithi053.stdout: "num_blocked_ops": 0 2024-09-13T05:03:51.678 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:51.687 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-13T05:03:51.730 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:51.730+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "size": 20, 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 600, 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "ops": [ 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1992 3.8 3:177631ee:::rbd_data.117cdd702390.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.276718689999996, 2024-09-13T05:03:51.796 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024146402000000001, 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1992 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.797 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517894+0000", 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517893+0000", 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999986 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517900+0000", 2024-09-13T05:03:51.798 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.4449999999999999e-06 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517916+0000", 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.6124e-05 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517922+0000", 2024-09-13T05:03:51.799 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0599999999999996e-06 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.517976+0000", 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.3324e-05 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518117+0000", 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00014149 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.800 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.519795+0000", 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.0016780670000000001 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541956+0000", 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.022160239000000002 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541964+0000", 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.3790000000000006e-06 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.801 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.541997+0000", 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.2728999999999998e-05 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542041+0000", 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.3900999999999999e-05 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.802 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:1993 3.8 3:129d6d6c:::rbd_data.117cdd702390.000000000000008c:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 97.276547385000001, 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024109009000000001, 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 1993 2024-09-13T05:03:51.803 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.804 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518065+0000", 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518069+0000", 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.4999999999999999e-06 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518074+0000", 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.7559999999999996e-06 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.805 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.518078+0000", 2024-09-13T05:03:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.653e-06 2024-09-13T05:03:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.806 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528320+0000", 2024-09-13T05:03:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.010242146000000001 2024-09-13T05:03:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.807 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.528456+0000", 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00013576899999999999 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "waiting for subops from 1", 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.530554+0000", 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.002098131 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "op_commit", 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542048+0000", 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.011493592 2024-09-13T05:03:51.808 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "sub_op_commit_rec", 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542085+0000", 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.7641000000000001e-05 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "commit_sent", 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542136+0000", 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.0506000000000001e-05 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:14.542174+0000", 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.8754999999999997e-05 2024-09-13T05:03:51.809 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74418 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 104448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.349607141999996, 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023607046999999999, 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.810 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74418 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445006+0000", 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.811 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445005+0000", 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445007+0000", 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.57e-06 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445010+0000", 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8370000000000001e-06 2024-09-13T05:03:51.812 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445012+0000", 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3470000000000001e-06 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468428+0000", 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023415735 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.813 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468517+0000", 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.9163e-05 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468613+0000", 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.5766000000000004e-05 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.814 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74419 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 241664~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.349597363000001, 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023722686, 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74419 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.815 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.816 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445015+0000", 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445017+0000", 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3459999999999999e-06 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445019+0000", 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8080000000000001e-06 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.817 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445022+0000", 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.407e-06 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468652+0000", 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023630254999999999 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468697+0000", 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5380000000000003e-05 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.818 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468738+0000", 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0706e-05 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74426 3.2 3:47226263:::rbd_data.117cb402b822.0000000000000001:1d [read 30720~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.349583324999998, 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023933494999999999, 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.819 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74426 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.820 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445030+0000", 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445029+0000", 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445031+0000", 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.7579999999999999e-06 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.821 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445034+0000", 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.039e-06 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445037+0000", 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6000000000000001e-06 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.822 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468771+0000", 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023734393999999999 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468866+0000", 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.4878999999999997e-05 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468963+0000", 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.7075000000000001e-05 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.823 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74435 3.2 3:4732e2b6:::rbd_data.117cb402b822.0000000000000041:1d [read 202752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.349557159, 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024030073999999998, 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74435 2024-09-13T05:03:51.824 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445056+0000", 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.825 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445055+0000", 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445057+0000", 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.4050000000000001e-06 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445059+0000", 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.6989999999999999e-06 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.826 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.445063+0000", 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6090000000000001e-06 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.468990+0000", 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023927187999999999 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469046+0000", 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5393999999999999e-05 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.827 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.469086+0000", 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.0110000000000001e-05 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74852 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.244678706999998, 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023498377000000001, 2024-09-13T05:03:51.828 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74852 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.829 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549934+0000", 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549936+0000", 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3499999999999999e-06 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.830 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549942+0000", 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5690000000000004e-06 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.549946+0000", 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6460000000000001e-06 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573282+0000", 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023335971 2024-09-13T05:03:51.831 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573355+0000", 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.3090999999999995e-05 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.573433+0000", 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.7169000000000006e-05 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.832 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74883 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4063232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.242424671999999, 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023559050000000002, 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74883 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.833 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552188+0000", 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.834 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552190+0000", 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2680000000000001e-06 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552196+0000", 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6710000000000004e-06 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552200+0000", 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.8250000000000004e-06 2024-09-13T05:03:51.835 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575659+0000", 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023458762000000001 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575695+0000", 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.5834999999999998e-05 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.836 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575747+0000", 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2123000000000002e-05 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74884 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4186112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.242408298000001, 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023776981999999999, 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.837 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74884 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552205+0000", 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.838 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552204+0000", 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552217+0000", 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.2507000000000001e-05 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552223+0000", 2024-09-13T05:03:51.839 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.4629999999999998e-06 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.552228+0000", 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.2889999999999997e-06 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575814+0000", 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023585386 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.840 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575896+0000", 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2373000000000006e-05 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.575982+0000", 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5292999999999998e-05 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.841 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74907 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3088384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.240578206999999, 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023582677999999999, 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74907 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.842 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554035+0000", 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554034+0000", 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.843 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554036+0000", 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.263e-06 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554039+0000", 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8600000000000001e-06 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554044+0000", 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5929999999999996e-06 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.844 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577427+0000", 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023383107 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577516+0000", 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.8975999999999996e-05 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577617+0000", 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.00010108800000000001 2024-09-13T05:03:51.845 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74908 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3211264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.240553083000002, 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023713964000000001, 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.846 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74908 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554060+0000", 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.847 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554059+0000", 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554061+0000", 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.0669999999999999e-06 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554065+0000", 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.6739999999999999e-06 2024-09-13T05:03:51.848 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554068+0000", 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.7800000000000001e-06 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577650+0000", 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023581775999999999 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577718+0000", 2024-09-13T05:03:51.849 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.8078999999999997e-05 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577774+0000", 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.6023000000000002e-05 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74909 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3334144~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.240542681000001, 2024-09-13T05:03:51.850 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023913496999999999, 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74909 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.851 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554070+0000", 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554071+0000", 2024-09-13T05:03:51.852 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.144e-06 2024-09-13T05:03:51.853 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.853 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.853 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.857 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554074+0000", 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.8899999999999999e-06 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.554076+0000", 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.244e-06 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577826+0000", 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023749749000000001 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.858 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577911+0000", 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5280999999999999e-05 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.577984+0000", 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2390000000000003e-05 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.859 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74937 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2850816~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.238224314999997, 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023486904999999999, 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74937 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.860 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556389+0000", 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556388+0000", 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.861 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556396+0000", 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.379e-06 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556402+0000", 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.3280000000000002e-06 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556406+0000", 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5589999999999999e-06 2024-09-13T05:03:51.862 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579700+0000", 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023293081 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579790+0000", 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0459999999999998e-05 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579875+0000", 2024-09-13T05:03:51.863 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.5434999999999998e-05 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74938 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 2973696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.238199731999998, 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023573340000000002, 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.864 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74938 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.865 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556413+0000", 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556421+0000", 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2349999999999999e-06 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556427+0000", 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0889999999999996e-06 2024-09-13T05:03:51.866 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556432+0000", 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9060000000000001e-06 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579892+0000", 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023459556999999999 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.867 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579937+0000", 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.5887999999999998e-05 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.579986+0000", 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.9039999999999998e-05 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74939 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3096576~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:51.868 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.238116769000001, 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023687123000000001, 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74939 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.869 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556496+0000", 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.870 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556499+0000", 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1779999999999999e-06 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556506+0000", 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6529999999999994e-06 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556512+0000", 2024-09-13T05:03:51.871 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.4559999999999999e-06 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580022+0000", 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023509768 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580105+0000", 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.2909000000000002e-05 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.872 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580183+0000", 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.8634000000000006e-05 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74940 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3219456~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:51.873 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.238096188, 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023773269999999999, 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74940 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:51.874 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556517+0000", 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556516+0000", 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.875 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556519+0000", 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3379999999999999e-06 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556525+0000", 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.0290000000000002e-06 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.876 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556529+0000", 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6650000000000002e-06 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580199+0000", 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023669577000000001 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580243+0000", 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.4678999999999997e-05 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.877 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580290+0000", 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6459999999999999e-05 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74941 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3342336~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.238079614, 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023966708999999999, 2024-09-13T05:03:51.878 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74941 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.879 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556533+0000", 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556535+0000", 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.2290000000000002e-06 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.880 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556541+0000", 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 6.054e-06 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556546+0000", 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6450000000000004e-06 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580325+0000", 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023779188999999999 2024-09-13T05:03:51.881 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580409+0000", 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4301999999999993e-05 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580500+0000", 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 9.0651999999999999e-05 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.882 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74942 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3465216~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.238062616999997, 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.024128163000000001, 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74942 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.883 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556550+0000", 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.884 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556552+0000", 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.21e-06 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556558+0000", 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.8969999999999996e-06 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.556563+0000", 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4.6249999999999998e-06 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.885 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580530+0000", 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023967769999999999 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580602+0000", 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.1264000000000001e-05 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.580678+0000", 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.6777e-05 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.886 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74946 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 3956736~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.236765931999997, 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023658351000000001, 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74946 2024-09-13T05:03:51.887 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.888 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557847+0000", 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967296 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557848+0000", 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.3599999999999999e-06 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557854+0000", 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.9349999999999999e-06 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.889 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557861+0000", 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 7.2649999999999999e-06 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581334+0000", 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023472815000000001 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581420+0000", 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.6250999999999998e-05 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.890 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581505+0000", 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 8.4926000000000006e-05 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "description": "osd_op(client.4476.0:74947 3.2 3:457088bc:::rbd_data.117c15713343.0000000000000001:20 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "initiated_at": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "age": 55.236749048, 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023778819, 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "type_data": { 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "flag_point": "started", 2024-09-13T05:03:51.891 INFO:teuthology.orchestra.run.smithi053.stdout: "client_info": { 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client": "client.4476", 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "client_addr": "172.21.15.53:0/861457407", 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "tid": 74947 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "events": [ 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "initiated", 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "throttled", 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:51.892 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "header_read", 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557864+0000", 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 4294967295.9999995 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "all_read", 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557865+0000", 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 1.491e-06 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "dispatched", 2024-09-13T05:03:51.893 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557868+0000", 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 3.1599999999999998e-06 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "queued_for_pg", 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.557871+0000", 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 2.3460000000000001e-06 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "reached_pg", 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581536+0000", 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 0.023665094000000001 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "started", 2024-09-13T05:03:51.894 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581592+0000", 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.5996999999999999e-05 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: }, 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: { 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "event": "done", 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "time": "2024-09-13T05:02:56.581643+0000", 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: "duration": 5.1004999999999997e-05 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: } 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout: ] 2024-09-13T05:03:51.895 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:03:51.897 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-13T05:03:51.897 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:51.896+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:51.920 INFO:teuthology.orchestra.run.smithi053.stderr:2024-09-13T05:03:51.919+0000 7f71977fe640 0 --2- 172.21.15.53:0/2441641686 >> [v2:172.21.15.53:3301/0,v1:172.21.15.53:6790/0] conn(0x7f71980af390 0x7f7198144f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-13T05:03:51.939 INFO:teuthology.orchestra.run.smithi053.stdout:osd.0: {} 2024-09-13T05:03:51.940 INFO:teuthology.orchestra.run.smithi053.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-13T05:03:51.956 INFO:teuthology.orchestra.run.smithi053.stdout:osd.1: {} 2024-09-13T05:03:51.956 INFO:teuthology.orchestra.run.smithi053.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-13T05:03:51.973 INFO:teuthology.orchestra.run.smithi053.stdout:osd.2: {} 2024-09-13T05:03:51.973 INFO:teuthology.orchestra.run.smithi053.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-13T05:03:51.992 INFO:teuthology.orchestra.run.smithi053.stdout:osd.3: {} 2024-09-13T05:03:51.993 INFO:teuthology.orchestra.run.smithi053.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-13T05:03:51.998 INFO:tasks.ceph.osd.7.smithi110.stderr:2024-09-13T05:03:51.997+0000 7fc95a887640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33009) UID: 0 2024-09-13T05:03:52.012 INFO:teuthology.orchestra.run.smithi053.stdout:osd.4: {} 2024-09-13T05:03:52.012 INFO:teuthology.orchestra.run.smithi053.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-13T05:03:52.019 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.020 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.029 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-13T05:03:52.030 INFO:teuthology.orchestra.run.smithi053.stdout:osd.5: {} 2024-09-13T05:03:52.030 INFO:teuthology.orchestra.run.smithi053.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-13T05:03:52.046 INFO:teuthology.orchestra.run.smithi053.stdout:osd.6: {} 2024-09-13T05:03:52.047 INFO:teuthology.orchestra.run.smithi053.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-13T05:03:52.063 INFO:teuthology.orchestra.run.smithi053.stdout:osd.7: {} 2024-09-13T05:03:52.064 INFO:teuthology.orchestra.run.smithi053.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-13T05:03:52.113 INFO:tasks.ceph.osd.4.smithi110.stderr:2024-09-13T05:03:52.113+0000 7f6114eb4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33001) UID: 0 2024-09-13T05:03:52.173 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.173 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.182 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-13T05:03:52.199 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:52.199+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:52.301 INFO:tasks.ceph.osd.2.smithi053.stderr:2024-09-13T05:03:52.300+0000 7f495fafd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33076) UID: 0 2024-09-13T05:03:52.325 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.325 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.333 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-13T05:03:52.402 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:52.401+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:52.435 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.436 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.444 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-13T05:03:52.502 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:52.502+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:52.545 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.545 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.554 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-13T05:03:52.603 INFO:tasks.ceph.osd.0.smithi053.stderr:2024-09-13T05:03:52.603+0000 7f0a99a9c640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33064) UID: 0 2024-09-13T05:03:52.655 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.656 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.664 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-13T05:03:52.704 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:52.704+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:52.766 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.766 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.775 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-13T05:03:52.818 INFO:tasks.ceph.osd.6.smithi110.stderr:2024-09-13T05:03:52.818+0000 7f95b9ed9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-13T05:03:52.880 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:52.880 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:52.886 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-13T05:03:52.906 INFO:tasks.ceph.osd.1.smithi053.stderr:2024-09-13T05:03:52.906+0000 7fb48eed1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33074) UID: 0 2024-09-13T05:03:52.981 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:52.982 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-13T05:03:52.982 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-13T05:03:52.982 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-13T05:03:52.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-13T05:03:53.027 INFO:tasks.ceph.osd.3.smithi053.stderr:2024-09-13T05:03:53.027+0000 7f70e56d1640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33084) UID: 0 2024-09-13T05:03:53.035 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:53.035 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:53.045 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-13T05:03:53.064 INFO:teuthology.orchestra.run.smithi053.stdout:All operations completed A-OK! 2024-09-13T05:03:53.077 DEBUG:teuthology.parallel:result is None 2024-09-13T05:03:53.080 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-13T05:03:53.090 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-13T05:03:53.102 INFO:tasks.thrashosds:joining thrashosds 2024-09-13T05:03:53.149 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:53.149 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:53.158 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-13T05:03:53.260 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:53.260 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:53.269 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-13T05:03:53.298 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-13T05:03:53.297+0000 7fd21ef37640 -1 mon.a@0(leader).osd e109 definitely_dead 0 2024-09-13T05:03:53.371 INFO:teuthology.orchestra.run.smithi110.stdout:ERROR: (22) Invalid argument 2024-09-13T05:03:53.371 INFO:teuthology.orchestra.run.smithi110.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-13T05:03:53.986 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is set 2024-09-13T05:03:53.986 INFO:teuthology.orchestra.run.smithi053.stderr:marked out osd.3. 2024-09-13T05:03:56.008 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-13T05:03:56.889 INFO:teuthology.orchestra.run.smithi053.stderr:noscrub is unset 2024-09-13T05:03:56.906 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-13T05:03:57.891 INFO:teuthology.orchestra.run.smithi053.stderr:nodeep-scrub is unset 2024-09-13T05:03:59.009 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-13T05:03:59.009 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-13T05:03:59.009 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-13T05:03:59.320 INFO:tasks.ceph.mon.a.smithi053.stderr:2024-09-13T05:03:59.319+0000 7fd21ef37640 -1 mon.a@0(leader).osd e114 definitely_dead 0 2024-09-13T05:03:59.886 INFO:teuthology.orchestra.run.smithi053.stderr:marked in osd.3. 2024-09-13T05:03:59.906 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-13T05:03:59.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-13T05:04:00.028 INFO:teuthology.orchestra.run.smithi053.stdout:mon.a: {} 2024-09-13T05:04:00.028 INFO:teuthology.orchestra.run.smithi053.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:04:00.039 INFO:teuthology.orchestra.run.smithi053.stdout:mon.b: {} 2024-09-13T05:04:00.039 INFO:teuthology.orchestra.run.smithi053.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:04:00.047 INFO:teuthology.orchestra.run.smithi053.stdout:mon.c: {} 2024-09-13T05:04:00.047 INFO:teuthology.orchestra.run.smithi053.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-13T05:04:00.058 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-13T05:04:00.888 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.0 to 1 (10000) 2024-09-13T05:04:00.907 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-13T05:04:01.890 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-13T05:04:01.910 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-13T05:04:02.912 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.1 to 1 (10000) 2024-09-13T05:04:02.929 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-13T05:04:03.913 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-13T05:04:03.931 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-13T05:04:04.918 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.2 to 1 (10000) 2024-09-13T05:04:04.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-13T05:04:05.919 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-13T05:04:05.935 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-13T05:04:06.924 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.3 to 1 (10000) 2024-09-13T05:04:06.941 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-13T05:04:07.929 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-13T05:04:07.946 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-13T05:04:08.935 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.4 to 1 (10000) 2024-09-13T05:04:08.951 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-13T05:04:09.940 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-13T05:04:09.957 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-13T05:04:10.946 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.5 to 1 (10000) 2024-09-13T05:04:10.962 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-13T05:04:11.951 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-13T05:04:11.968 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-13T05:04:12.957 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.6 to 1 (10000) 2024-09-13T05:04:12.973 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-13T05:04:13.963 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-13T05:04:13.981 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-13T05:04:14.967 INFO:teuthology.orchestra.run.smithi053.stderr:reweighted osd.7 to 1 (10000) 2024-09-13T05:04:14.984 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-13T05:04:15.972 INFO:teuthology.orchestra.run.smithi053.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-13T05:04:15.990 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-13T05:04:15.990 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-13T05:04:15.990 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-13T05:04:15.991 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-13T05:04:15.991 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-13T05:04:15.991 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:04:16.287 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:16.288 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":131,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:04:15.967833+0000","last_up_change":"2024-09-13T05:01:55.160352+0000","last_in_change":"2024-09-13T05:03:59.880985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T05:01:55.817858+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T05:01:55.981425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T05:02:05.508713+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6817","nonce":1769628429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6819","nonce":1769628429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6823","nonce":1769628429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6821","nonce":1769628429}]},"public_addr":"172.21.15.53:6817/1769628429","cluster_addr":"172.21.15.53:6819/1769628429","heartbeat_back_addr":"172.21.15.53:6823/1769628429","heartbeat_front_addr":"172.21.15.53:6821/1769628429","state":["exists","up"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6801","nonce":827277153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6803","nonce":827277153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6807","nonce":827277153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6805","nonce":827277153}]},"public_addr":"172.21.15.53:6801/827277153","cluster_addr":"172.21.15.53:6803/827277153","heartbeat_back_addr":"172.21.15.53:6807/827277153","heartbeat_front_addr":"172.21.15.53:6805/827277153","state":["exists","up"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6809","nonce":1334123504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6811","nonce":1334123504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6815","nonce":1334123504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6813","nonce":1334123504}]},"public_addr":"172.21.15.53:6809/1334123504","cluster_addr":"172.21.15.53:6811/1334123504","heartbeat_back_addr":"172.21.15.53:6815/1334123504","heartbeat_front_addr":"172.21.15.53:6813/1334123504","state":["exists","up"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6825","nonce":4134389313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6827","nonce":4134389313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6831","nonce":4134389313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6829","nonce":4134389313}]},"public_addr":"172.21.15.53:6825/4134389313","cluster_addr":"172.21.15.53:6827/4134389313","heartbeat_back_addr":"172.21.15.53:6831/4134389313","heartbeat_front_addr":"172.21.15.53:6829/4134389313","state":["exists","up"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6809","nonce":3681797816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6811","nonce":3681797816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6815","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6816","nonce":3681797816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6813","nonce":3681797816}]},"public_addr":"172.21.15.110:6809/3681797816","cluster_addr":"172.21.15.110:6811/3681797816","heartbeat_back_addr":"172.21.15.110:6816/3681797816","heartbeat_front_addr":"172.21.15.110:6813/3681797816","state":["exists","up"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6801","nonce":81482652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6803","nonce":81482652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6807","nonce":81482652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6805","nonce":81482652}]},"public_addr":"172.21.15.110:6801/81482652","cluster_addr":"172.21.15.110:6803/81482652","heartbeat_back_addr":"172.21.15.110:6807/81482652","heartbeat_front_addr":"172.21.15.110:6805/81482652","state":["exists","up"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6817","nonce":576600432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6819","nonce":576600432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6823","nonce":576600432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6821","nonce":576600432}]},"public_addr":"172.21.15.110:6817/576600432","cluster_addr":"172.21.15.110:6819/576600432","heartbeat_back_addr":"172.21.15.110:6823/576600432","heartbeat_front_addr":"172.21.15.110:6821/576600432","state":["exists","up"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6825","nonce":896779674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6827","nonce":896779674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6831","nonce":896779674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6829","nonce":896779674}]},"public_addr":"172.21.15.110:6825/896779674","cluster_addr":"172.21.15.110:6827/896779674","heartbeat_back_addr":"172.21.15.110:6831/896779674","heartbeat_front_addr":"172.21.15.110:6829/896779674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358655+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.343203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.307081+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.315153+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.280902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.302500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.354685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:04:16.301 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-13T05:04:16.301 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:04:16.588 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:16.588 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":131,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:04:15.967833+0000","last_up_change":"2024-09-13T05:01:55.160352+0000","last_in_change":"2024-09-13T05:03:59.880985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T05:01:55.817858+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T05:01:55.981425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T05:02:05.508713+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6817","nonce":1769628429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6819","nonce":1769628429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6823","nonce":1769628429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6821","nonce":1769628429}]},"public_addr":"172.21.15.53:6817/1769628429","cluster_addr":"172.21.15.53:6819/1769628429","heartbeat_back_addr":"172.21.15.53:6823/1769628429","heartbeat_front_addr":"172.21.15.53:6821/1769628429","state":["exists","up"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6801","nonce":827277153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6803","nonce":827277153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6807","nonce":827277153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6805","nonce":827277153}]},"public_addr":"172.21.15.53:6801/827277153","cluster_addr":"172.21.15.53:6803/827277153","heartbeat_back_addr":"172.21.15.53:6807/827277153","heartbeat_front_addr":"172.21.15.53:6805/827277153","state":["exists","up"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6809","nonce":1334123504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6811","nonce":1334123504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6815","nonce":1334123504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6813","nonce":1334123504}]},"public_addr":"172.21.15.53:6809/1334123504","cluster_addr":"172.21.15.53:6811/1334123504","heartbeat_back_addr":"172.21.15.53:6815/1334123504","heartbeat_front_addr":"172.21.15.53:6813/1334123504","state":["exists","up"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6825","nonce":4134389313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6827","nonce":4134389313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6831","nonce":4134389313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6829","nonce":4134389313}]},"public_addr":"172.21.15.53:6825/4134389313","cluster_addr":"172.21.15.53:6827/4134389313","heartbeat_back_addr":"172.21.15.53:6831/4134389313","heartbeat_front_addr":"172.21.15.53:6829/4134389313","state":["exists","up"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6809","nonce":3681797816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6811","nonce":3681797816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6815","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6816","nonce":3681797816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6813","nonce":3681797816}]},"public_addr":"172.21.15.110:6809/3681797816","cluster_addr":"172.21.15.110:6811/3681797816","heartbeat_back_addr":"172.21.15.110:6816/3681797816","heartbeat_front_addr":"172.21.15.110:6813/3681797816","state":["exists","up"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6801","nonce":81482652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6803","nonce":81482652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6807","nonce":81482652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6805","nonce":81482652}]},"public_addr":"172.21.15.110:6801/81482652","cluster_addr":"172.21.15.110:6803/81482652","heartbeat_back_addr":"172.21.15.110:6807/81482652","heartbeat_front_addr":"172.21.15.110:6805/81482652","state":["exists","up"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6817","nonce":576600432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6819","nonce":576600432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6823","nonce":576600432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6821","nonce":576600432}]},"public_addr":"172.21.15.110:6817/576600432","cluster_addr":"172.21.15.110:6819/576600432","heartbeat_back_addr":"172.21.15.110:6823/576600432","heartbeat_front_addr":"172.21.15.110:6821/576600432","state":["exists","up"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6825","nonce":896779674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6827","nonce":896779674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6831","nonce":896779674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6829","nonce":896779674}]},"public_addr":"172.21.15.110:6825/896779674","cluster_addr":"172.21.15.110:6827/896779674","heartbeat_back_addr":"172.21.15.110:6831/896779674","heartbeat_front_addr":"172.21.15.110:6829/896779674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358655+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.343203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.307081+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.315153+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.280902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.302500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.354685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:04:16.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-13T05:04:16.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-13T05:04:16.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-13T05:04:16.602 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-13T05:04:16.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-13T05:04:16.603 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-13T05:04:16.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-13T05:04:16.604 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-13T05:04:16.854 INFO:teuthology.orchestra.run.smithi053.stdout:55834574881 2024-09-13T05:04:16.854 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T05:04:16.855 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.856 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T05:04:16.857 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.857 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T05:04:16.858 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.858 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T05:04:16.859 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.859 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T05:04:16.860 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.861 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.861 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T05:04:16.861 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:16.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T05:04:16.862 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T05:04:17.423 INFO:teuthology.orchestra.run.smithi053.stdout:55834574879 2024-09-13T05:04:17.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574879 for osd.1 2024-09-13T05:04:17.481 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.483 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.493 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.497 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.499 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.502 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.7 2024-09-13T05:04:17.503 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.4 2024-09-13T05:04:17.509 INFO:teuthology.orchestra.run.smithi053.stdout:55834574878 2024-09-13T05:04:17.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.3 2024-09-13T05:04:17.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.5 2024-09-13T05:04:17.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.0 2024-09-13T05:04:17.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.2 2024-09-13T05:04:17.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574878 for osd.6 2024-09-13T05:04:18.443 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-13T05:04:18.503 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-13T05:04:18.506 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-13T05:04:18.514 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-13T05:04:18.517 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-13T05:04:18.520 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-13T05:04:18.524 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-13T05:04:18.526 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-13T05:04:18.953 INFO:teuthology.orchestra.run.smithi053.stdout:55834574881 2024-09-13T05:04:18.971 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574881 for osd.1 2024-09-13T05:04:18.971 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.052 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.3 2024-09-13T05:04:19.065 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.076 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.095 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.7 2024-09-13T05:04:19.095 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.116 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.2 2024-09-13T05:04:19.129 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.143 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.148 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.154 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.0 2024-09-13T05:04:19.157 INFO:teuthology.orchestra.run.smithi053.stdout:55834574880 2024-09-13T05:04:19.158 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.4 2024-09-13T05:04:19.163 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.6 2024-09-13T05:04:19.169 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574880 got 55834574880 for osd.5 2024-09-13T05:04:19.170 DEBUG:teuthology.parallel:result is None 2024-09-13T05:04:19.170 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-13T05:04:19.171 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:19.500 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:19.501 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:19.512 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-13T05:04:17.962917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448336,"kb_used_data":12360,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526384,"statfs":{"total":773094113280,"available":772635017216,"internally_reserved":0,"allocated":12656640,"data_stored":7180584,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002789"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2584,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966719+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:14.966719+0000","last_peered":"2024-09-13T05:04:14.966719+0000","last_clean":"2024-09-13T05:04:14.966719+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:14.966719+0000","last_undegraded":"2024-09-13T05:04:14.966719+0000","last_fullsized":"2024-09-13T05:04:14.966719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2886,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966338+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:14.966338+0000","last_peered":"2024-09-13T05:04:14.966338+0000","last_clean":"2024-09-13T05:04:14.966338+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:14.966338+0000","last_undegraded":"2024-09-13T05:04:14.966338+0000","last_fullsized":"2024-09-13T05:04:14.966338+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2664,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965712+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:14.965712+0000","last_peered":"2024-09-13T05:04:14.965712+0000","last_clean":"2024-09-13T05:04:14.965712+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:14.965712+0000","last_undegraded":"2024-09-13T05:04:14.965712+0000","last_fullsized":"2024-09-13T05:04:14.965712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5493,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965611+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:14.965611+0000","last_peered":"2024-09-13T05:04:14.965611+0000","last_clean":"2024-09-13T05:04:14.965611+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:14.965611+0000","last_undegraded":"2024-09-13T05:04:14.965611+0000","last_fullsized":"2024-09-13T05:04:14.965611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5545,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966737+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:14.966737+0000","last_peered":"2024-09-13T05:04:14.966737+0000","last_clean":"2024-09-13T05:04:14.966737+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:14.966737+0000","last_undegraded":"2024-09-13T05:04:14.966737+0000","last_fullsized":"2024-09-13T05:04:14.966737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6375,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970825+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:14.970825+0000","last_peered":"2024-09-13T05:04:14.970825+0000","last_clean":"2024-09-13T05:04:14.970825+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:14.970825+0000","last_undegraded":"2024-09-13T05:04:14.970825+0000","last_fullsized":"2024-09-13T05:04:14.970825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2142,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772939+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.772939+0000","last_peered":"2024-09-13T05:04:15.772939+0000","last_clean":"2024-09-13T05:04:15.772939+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.772939+0000","last_undegraded":"2024-09-13T05:04:15.772939+0000","last_fullsized":"2024-09-13T05:04:15.772939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2678,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966236+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:14.966236+0000","last_peered":"2024-09-13T05:04:14.966236+0000","last_clean":"2024-09-13T05:04:14.966236+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:14.966236+0000","last_undegraded":"2024-09-13T05:04:14.966236+0000","last_fullsized":"2024-09-13T05:04:14.966236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2793,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966774+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:14.966774+0000","last_peered":"2024-09-13T05:04:14.966774+0000","last_clean":"2024-09-13T05:04:14.966774+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:14.966774+0000","last_undegraded":"2024-09-13T05:04:14.966774+0000","last_fullsized":"2024-09-13T05:04:14.966774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3276,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965730+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:14.965730+0000","last_peered":"2024-09-13T05:04:14.965730+0000","last_clean":"2024-09-13T05:04:14.965730+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:14.965730+0000","last_undegraded":"2024-09-13T05:04:14.965730+0000","last_fullsized":"2024-09-13T05:04:14.965730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3681,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170347+0000","last_change":"2024-09-13T05:04:03.921625+0000","last_active":"2024-09-13T05:04:10.170347+0000","last_peered":"2024-09-13T05:04:10.170347+0000","last_clean":"2024-09-13T05:04:10.170347+0000","last_became_active":"2024-09-13T05:04:03.919391+0000","last_became_peered":"2024-09-13T05:04:03.919391+0000","last_unstale":"2024-09-13T05:04:10.170347+0000","last_undegraded":"2024-09-13T05:04:10.170347+0000","last_fullsized":"2024-09-13T05:04:10.170347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:04.622268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002304637,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2137,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970845+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:14.970845+0000","last_peered":"2024-09-13T05:04:14.970845+0000","last_clean":"2024-09-13T05:04:14.970845+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:14.970845+0000","last_undegraded":"2024-09-13T05:04:14.970845+0000","last_fullsized":"2024-09-13T05:04:14.970845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118273+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.118273+0000","last_peered":"2024-09-13T05:04:16.118273+0000","last_clean":"2024-09-13T05:04:16.118273+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.118273+0000","last_undegraded":"2024-09-13T05:04:16.118273+0000","last_fullsized":"2024-09-13T05:04:16.118273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3183,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965786+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:14.965786+0000","last_peered":"2024-09-13T05:04:14.965786+0000","last_clean":"2024-09-13T05:04:14.965786+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:14.965786+0000","last_undegraded":"2024-09-13T05:04:14.965786+0000","last_fullsized":"2024-09-13T05:04:14.965786+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118261+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.118261+0000","last_peered":"2024-09-13T05:04:16.118261+0000","last_clean":"2024-09-13T05:04:16.118261+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.118261+0000","last_undegraded":"2024-09-13T05:04:16.118261+0000","last_fullsized":"2024-09-13T05:04:16.118261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4990,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772993+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.772993+0000","last_peered":"2024-09-13T05:04:15.772993+0000","last_clean":"2024-09-13T05:04:15.772993+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.772993+0000","last_undegraded":"2024-09-13T05:04:15.772993+0000","last_fullsized":"2024-09-13T05:04:15.772993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970912+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:14.970912+0000","last_peered":"2024-09-13T05:04:14.970912+0000","last_clean":"2024-09-13T05:04:14.970912+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:14.970912+0000","last_undegraded":"2024-09-13T05:04:14.970912+0000","last_fullsized":"2024-09-13T05:04:14.970912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2550,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.967026+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:14.967026+0000","last_peered":"2024-09-13T05:04:14.967026+0000","last_clean":"2024-09-13T05:04:14.967026+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:14.967026+0000","last_undegraded":"2024-09-13T05:04:14.967026+0000","last_fullsized":"2024-09-13T05:04:14.967026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966392+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:14.966392+0000","last_peered":"2024-09-13T05:04:14.966392+0000","last_clean":"2024-09-13T05:04:14.966392+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:14.966392+0000","last_undegraded":"2024-09-13T05:04:14.966392+0000","last_fullsized":"2024-09-13T05:04:14.966392+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3463,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974508+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:15.974508+0000","last_peered":"2024-09-13T05:04:15.974508+0000","last_clean":"2024-09-13T05:04:15.974508+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:15.974508+0000","last_undegraded":"2024-09-13T05:04:15.974508+0000","last_fullsized":"2024-09-13T05:04:15.974508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2588,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966269+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:14.966269+0000","last_peered":"2024-09-13T05:04:14.966269+0000","last_clean":"2024-09-13T05:04:14.966269+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:14.966269+0000","last_undegraded":"2024-09-13T05:04:14.966269+0000","last_fullsized":"2024-09-13T05:04:14.966269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":220,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.971997+0000","last_change":"2024-09-13T05:04:15.971997+0000","last_active":"2024-09-13T05:04:15.971997+0000","last_peered":"2024-09-13T05:03:17.966011+0000","last_clean":"2024-09-13T05:03:17.966011+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:04:15.971997+0000","last_undegraded":"2024-09-13T05:04:15.971997+0000","last_fullsized":"2024-09-13T05:04:15.971997+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":132,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772940+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.772940+0000","last_peered":"2024-09-13T05:04:15.772940+0000","last_clean":"2024-09-13T05:04:15.772940+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.772940+0000","last_undegraded":"2024-09-13T05:04:15.772940+0000","last_fullsized":"2024-09-13T05:04:15.772940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4224,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772922+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.772922+0000","last_peered":"2024-09-13T05:04:15.772922+0000","last_clean":"2024-09-13T05:04:15.772922+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.772922+0000","last_undegraded":"2024-09-13T05:04:15.772922+0000","last_fullsized":"2024-09-13T05:04:15.772922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":77,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972139+0000","last_change":"2024-09-13T05:04:15.972139+0000","last_active":"2024-09-13T05:04:15.972139+0000","last_peered":"2024-09-13T05:03:17.966055+0000","last_clean":"2024-09-13T05:03:17.966055+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:04:15.972139+0000","last_undegraded":"2024-09-13T05:04:15.972139+0000","last_fullsized":"2024-09-13T05:04:15.972139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3692,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972152+0000","last_change":"2024-09-13T05:04:15.972152+0000","last_active":"2024-09-13T05:04:15.972152+0000","last_peered":"2024-09-13T05:03:53.037707+0000","last_clean":"2024-09-13T05:03:53.037707+0000","last_became_active":"2024-09-13T05:03:51.976212+0000","last_became_peered":"2024-09-13T05:03:51.976212+0000","last_unstale":"2024-09-13T05:04:15.972152+0000","last_undegraded":"2024-09-13T05:04:15.972152+0000","last_fullsized":"2024-09-13T05:04:15.972152+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965596+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:14.965596+0000","last_peered":"2024-09-13T05:04:14.965596+0000","last_clean":"2024-09-13T05:04:14.965596+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:14.965596+0000","last_undegraded":"2024-09-13T05:04:14.965596+0000","last_fullsized":"2024-09-13T05:04:14.965596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3912,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966471+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:14.966471+0000","last_peered":"2024-09-13T05:04:14.966471+0000","last_clean":"2024-09-13T05:04:14.966471+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:14.966471+0000","last_undegraded":"2024-09-13T05:04:14.966471+0000","last_fullsized":"2024-09-13T05:04:14.966471+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972960+0000","last_change":"2024-09-13T05:03:18.975073+0000","last_active":"2024-09-13T05:01:56.162205+0000","last_peered":"2024-09-13T05:01:56.162205+0000","last_clean":"2024-09-13T05:01:56.162205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T05:04:15.972960+0000","last_undegraded":"2024-09-13T05:04:15.972960+0000","last_fullsized":"2024-09-13T05:04:15.972960+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3732,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966129+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:14.966129+0000","last_peered":"2024-09-13T05:04:14.966129+0000","last_clean":"2024-09-13T05:04:14.966129+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:14.966129+0000","last_undegraded":"2024-09-13T05:04:14.966129+0000","last_fullsized":"2024-09-13T05:04:14.966129+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2874,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975235+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:15.975235+0000","last_peered":"2024-09-13T05:04:15.975235+0000","last_clean":"2024-09-13T05:04:15.975235+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:15.975235+0000","last_undegraded":"2024-09-13T05:04:15.975235+0000","last_fullsized":"2024-09-13T05:04:15.975235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4658,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118235+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.118235+0000","last_peered":"2024-09-13T05:04:16.118235+0000","last_clean":"2024-09-13T05:04:16.118235+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.118235+0000","last_undegraded":"2024-09-13T05:04:16.118235+0000","last_fullsized":"2024-09-13T05:04:16.118235+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2827,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966283+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:14.966283+0000","last_peered":"2024-09-13T05:04:14.966283+0000","last_clean":"2024-09-13T05:04:14.966283+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:14.966283+0000","last_undegraded":"2024-09-13T05:04:14.966283+0000","last_fullsized":"2024-09-13T05:04:14.966283+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3800,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965595+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:14.965595+0000","last_peered":"2024-09-13T05:04:14.965595+0000","last_clean":"2024-09-13T05:04:14.965595+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:14.965595+0000","last_undegraded":"2024-09-13T05:04:14.965595+0000","last_fullsized":"2024-09-13T05:04:14.965595+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5312,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:04:09.035348+0000","last_change":"2024-09-13T05:03:52.987943+0000","last_active":"2024-09-13T05:04:09.035348+0000","last_peered":"2024-09-13T05:04:09.035348+0000","last_clean":"2024-09-13T05:04:09.035348+0000","last_became_active":"2024-09-13T05:03:19.880315+0000","last_became_peered":"2024-09-13T05:03:19.880315+0000","last_unstale":"2024-09-13T05:04:09.035348+0000","last_undegraded":"2024-09-13T05:04:09.035348+0000","last_fullsized":"2024-09-13T05:04:09.035348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:15:27.243701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038146700000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5621,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170587+0000","last_change":"2024-09-13T05:03:52.986411+0000","last_active":"2024-09-13T05:04:10.170587+0000","last_peered":"2024-09-13T05:04:10.170587+0000","last_clean":"2024-09-13T05:04:10.170587+0000","last_became_active":"2024-09-13T05:03:19.880257+0000","last_became_peered":"2024-09-13T05:03:19.880257+0000","last_unstale":"2024-09-13T05:04:10.170587+0000","last_undegraded":"2024-09-13T05:04:10.170587+0000","last_fullsized":"2024-09-13T05:04:10.170587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:07.964691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3028,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972483+0000","last_change":"2024-09-13T05:04:15.972483+0000","last_active":"2024-09-13T05:04:15.972483+0000","last_peered":"2024-09-13T05:03:51.973455+0000","last_clean":"2024-09-13T05:03:51.973455+0000","last_became_active":"2024-09-13T05:03:39.927143+0000","last_became_peered":"2024-09-13T05:03:39.927143+0000","last_unstale":"2024-09-13T05:04:15.972483+0000","last_undegraded":"2024-09-13T05:04:15.972483+0000","last_fullsized":"2024-09-13T05:04:15.972483+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3101,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.972664+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:14.972664+0000","last_peered":"2024-09-13T05:04:14.972664+0000","last_clean":"2024-09-13T05:04:14.972664+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:14.972664+0000","last_undegraded":"2024-09-13T05:04:14.972664+0000","last_fullsized":"2024-09-13T05:04:14.972664+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4252,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170402+0000","last_change":"2024-09-13T05:03:52.986811+0000","last_active":"2024-09-13T05:04:10.170402+0000","last_peered":"2024-09-13T05:04:10.170402+0000","last_clean":"2024-09-13T05:04:10.170402+0000","last_became_active":"2024-09-13T05:03:19.880261+0000","last_became_peered":"2024-09-13T05:03:19.880261+0000","last_unstale":"2024-09-13T05:04:10.170402+0000","last_undegraded":"2024-09-13T05:04:10.170402+0000","last_fullsized":"2024-09-13T05:04:10.170402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:14.799250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966628+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:14.966628+0000","last_peered":"2024-09-13T05:04:14.966628+0000","last_clean":"2024-09-13T05:04:14.966628+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:14.966628+0000","last_undegraded":"2024-09-13T05:04:14.966628+0000","last_fullsized":"2024-09-13T05:04:14.966628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3279,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.773336+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.773336+0000","last_peered":"2024-09-13T05:04:15.773336+0000","last_clean":"2024-09-13T05:04:15.773336+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.773336+0000","last_undegraded":"2024-09-13T05:04:15.773336+0000","last_fullsized":"2024-09-13T05:04:15.773336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1617920,"data_stored":1335977,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40836,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331004,"statfs":{"total":96636764160,"available":96594948096,"internally_reserved":0,"allocated":1556480,"data_stored":745590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.435},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.5}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.625}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68752,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303088,"statfs":{"total":96636764160,"available":96566362112,"internally_reserved":0,"allocated":1572864,"data_stored":751451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":3.2400000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":2.681}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":3.3730000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.4380000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":1.242},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.37}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.35699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68748,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303092,"statfs":{"total":96636764160,"available":96566366208,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.22900000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.371},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.309},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.435}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.502}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.40699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343284,"statfs":{"total":96636764160,"available":96607522816,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.318}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.623}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72904,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298936,"statfs":{"total":96636764160,"available":96562110464,"internally_reserved":0,"allocated":1630208,"data_stored":1341819,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.318},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58900,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312940,"statfs":{"total":96636764160,"available":96576450560,"internally_reserved":0,"allocated":1572864,"data_stored":751451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.501},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.629},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:19.515 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:19.800 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:19.801 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:19.812 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-13T05:04:17.962917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448336,"kb_used_data":12360,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526384,"statfs":{"total":773094113280,"available":772635017216,"internally_reserved":0,"allocated":12656640,"data_stored":7180584,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.002789"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2584,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966719+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:14.966719+0000","last_peered":"2024-09-13T05:04:14.966719+0000","last_clean":"2024-09-13T05:04:14.966719+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:14.966719+0000","last_undegraded":"2024-09-13T05:04:14.966719+0000","last_fullsized":"2024-09-13T05:04:14.966719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2886,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966338+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:14.966338+0000","last_peered":"2024-09-13T05:04:14.966338+0000","last_clean":"2024-09-13T05:04:14.966338+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:14.966338+0000","last_undegraded":"2024-09-13T05:04:14.966338+0000","last_fullsized":"2024-09-13T05:04:14.966338+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2664,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965712+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:14.965712+0000","last_peered":"2024-09-13T05:04:14.965712+0000","last_clean":"2024-09-13T05:04:14.965712+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:14.965712+0000","last_undegraded":"2024-09-13T05:04:14.965712+0000","last_fullsized":"2024-09-13T05:04:14.965712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5493,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965611+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:14.965611+0000","last_peered":"2024-09-13T05:04:14.965611+0000","last_clean":"2024-09-13T05:04:14.965611+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:14.965611+0000","last_undegraded":"2024-09-13T05:04:14.965611+0000","last_fullsized":"2024-09-13T05:04:14.965611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5545,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966737+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:14.966737+0000","last_peered":"2024-09-13T05:04:14.966737+0000","last_clean":"2024-09-13T05:04:14.966737+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:14.966737+0000","last_undegraded":"2024-09-13T05:04:14.966737+0000","last_fullsized":"2024-09-13T05:04:14.966737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6375,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970825+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:14.970825+0000","last_peered":"2024-09-13T05:04:14.970825+0000","last_clean":"2024-09-13T05:04:14.970825+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:14.970825+0000","last_undegraded":"2024-09-13T05:04:14.970825+0000","last_fullsized":"2024-09-13T05:04:14.970825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2142,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772939+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.772939+0000","last_peered":"2024-09-13T05:04:15.772939+0000","last_clean":"2024-09-13T05:04:15.772939+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.772939+0000","last_undegraded":"2024-09-13T05:04:15.772939+0000","last_fullsized":"2024-09-13T05:04:15.772939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2678,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966236+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:14.966236+0000","last_peered":"2024-09-13T05:04:14.966236+0000","last_clean":"2024-09-13T05:04:14.966236+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:14.966236+0000","last_undegraded":"2024-09-13T05:04:14.966236+0000","last_fullsized":"2024-09-13T05:04:14.966236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2793,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966774+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:14.966774+0000","last_peered":"2024-09-13T05:04:14.966774+0000","last_clean":"2024-09-13T05:04:14.966774+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:14.966774+0000","last_undegraded":"2024-09-13T05:04:14.966774+0000","last_fullsized":"2024-09-13T05:04:14.966774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3276,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965730+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:14.965730+0000","last_peered":"2024-09-13T05:04:14.965730+0000","last_clean":"2024-09-13T05:04:14.965730+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:14.965730+0000","last_undegraded":"2024-09-13T05:04:14.965730+0000","last_fullsized":"2024-09-13T05:04:14.965730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3681,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170347+0000","last_change":"2024-09-13T05:04:03.921625+0000","last_active":"2024-09-13T05:04:10.170347+0000","last_peered":"2024-09-13T05:04:10.170347+0000","last_clean":"2024-09-13T05:04:10.170347+0000","last_became_active":"2024-09-13T05:04:03.919391+0000","last_became_peered":"2024-09-13T05:04:03.919391+0000","last_unstale":"2024-09-13T05:04:10.170347+0000","last_undegraded":"2024-09-13T05:04:10.170347+0000","last_fullsized":"2024-09-13T05:04:10.170347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:04.622268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002304637,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2137,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970845+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:14.970845+0000","last_peered":"2024-09-13T05:04:14.970845+0000","last_clean":"2024-09-13T05:04:14.970845+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:14.970845+0000","last_undegraded":"2024-09-13T05:04:14.970845+0000","last_fullsized":"2024-09-13T05:04:14.970845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118273+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.118273+0000","last_peered":"2024-09-13T05:04:16.118273+0000","last_clean":"2024-09-13T05:04:16.118273+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.118273+0000","last_undegraded":"2024-09-13T05:04:16.118273+0000","last_fullsized":"2024-09-13T05:04:16.118273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3183,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965786+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:14.965786+0000","last_peered":"2024-09-13T05:04:14.965786+0000","last_clean":"2024-09-13T05:04:14.965786+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:14.965786+0000","last_undegraded":"2024-09-13T05:04:14.965786+0000","last_fullsized":"2024-09-13T05:04:14.965786+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118261+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.118261+0000","last_peered":"2024-09-13T05:04:16.118261+0000","last_clean":"2024-09-13T05:04:16.118261+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.118261+0000","last_undegraded":"2024-09-13T05:04:16.118261+0000","last_fullsized":"2024-09-13T05:04:16.118261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4990,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772993+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.772993+0000","last_peered":"2024-09-13T05:04:15.772993+0000","last_clean":"2024-09-13T05:04:15.772993+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.772993+0000","last_undegraded":"2024-09-13T05:04:15.772993+0000","last_fullsized":"2024-09-13T05:04:15.772993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970912+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:14.970912+0000","last_peered":"2024-09-13T05:04:14.970912+0000","last_clean":"2024-09-13T05:04:14.970912+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:14.970912+0000","last_undegraded":"2024-09-13T05:04:14.970912+0000","last_fullsized":"2024-09-13T05:04:14.970912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2550,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.967026+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:14.967026+0000","last_peered":"2024-09-13T05:04:14.967026+0000","last_clean":"2024-09-13T05:04:14.967026+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:14.967026+0000","last_undegraded":"2024-09-13T05:04:14.967026+0000","last_fullsized":"2024-09-13T05:04:14.967026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966392+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:14.966392+0000","last_peered":"2024-09-13T05:04:14.966392+0000","last_clean":"2024-09-13T05:04:14.966392+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:14.966392+0000","last_undegraded":"2024-09-13T05:04:14.966392+0000","last_fullsized":"2024-09-13T05:04:14.966392+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3463,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974508+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:15.974508+0000","last_peered":"2024-09-13T05:04:15.974508+0000","last_clean":"2024-09-13T05:04:15.974508+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:15.974508+0000","last_undegraded":"2024-09-13T05:04:15.974508+0000","last_fullsized":"2024-09-13T05:04:15.974508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2588,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966269+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:14.966269+0000","last_peered":"2024-09-13T05:04:14.966269+0000","last_clean":"2024-09-13T05:04:14.966269+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:14.966269+0000","last_undegraded":"2024-09-13T05:04:14.966269+0000","last_fullsized":"2024-09-13T05:04:14.966269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":220,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.971997+0000","last_change":"2024-09-13T05:04:15.971997+0000","last_active":"2024-09-13T05:04:15.971997+0000","last_peered":"2024-09-13T05:03:17.966011+0000","last_clean":"2024-09-13T05:03:17.966011+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:04:15.971997+0000","last_undegraded":"2024-09-13T05:04:15.971997+0000","last_fullsized":"2024-09-13T05:04:15.971997+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":132,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772940+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.772940+0000","last_peered":"2024-09-13T05:04:15.772940+0000","last_clean":"2024-09-13T05:04:15.772940+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.772940+0000","last_undegraded":"2024-09-13T05:04:15.772940+0000","last_fullsized":"2024-09-13T05:04:15.772940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4224,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772922+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.772922+0000","last_peered":"2024-09-13T05:04:15.772922+0000","last_clean":"2024-09-13T05:04:15.772922+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.772922+0000","last_undegraded":"2024-09-13T05:04:15.772922+0000","last_fullsized":"2024-09-13T05:04:15.772922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":77,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972139+0000","last_change":"2024-09-13T05:04:15.972139+0000","last_active":"2024-09-13T05:04:15.972139+0000","last_peered":"2024-09-13T05:03:17.966055+0000","last_clean":"2024-09-13T05:03:17.966055+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:04:15.972139+0000","last_undegraded":"2024-09-13T05:04:15.972139+0000","last_fullsized":"2024-09-13T05:04:15.972139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3692,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972152+0000","last_change":"2024-09-13T05:04:15.972152+0000","last_active":"2024-09-13T05:04:15.972152+0000","last_peered":"2024-09-13T05:03:53.037707+0000","last_clean":"2024-09-13T05:03:53.037707+0000","last_became_active":"2024-09-13T05:03:51.976212+0000","last_became_peered":"2024-09-13T05:03:51.976212+0000","last_unstale":"2024-09-13T05:04:15.972152+0000","last_undegraded":"2024-09-13T05:04:15.972152+0000","last_fullsized":"2024-09-13T05:04:15.972152+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965596+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:14.965596+0000","last_peered":"2024-09-13T05:04:14.965596+0000","last_clean":"2024-09-13T05:04:14.965596+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:14.965596+0000","last_undegraded":"2024-09-13T05:04:14.965596+0000","last_fullsized":"2024-09-13T05:04:14.965596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3912,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966471+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:14.966471+0000","last_peered":"2024-09-13T05:04:14.966471+0000","last_clean":"2024-09-13T05:04:14.966471+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:14.966471+0000","last_undegraded":"2024-09-13T05:04:14.966471+0000","last_fullsized":"2024-09-13T05:04:14.966471+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972960+0000","last_change":"2024-09-13T05:03:18.975073+0000","last_active":"2024-09-13T05:01:56.162205+0000","last_peered":"2024-09-13T05:01:56.162205+0000","last_clean":"2024-09-13T05:01:56.162205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T05:04:15.972960+0000","last_undegraded":"2024-09-13T05:04:15.972960+0000","last_fullsized":"2024-09-13T05:04:15.972960+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3732,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966129+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:14.966129+0000","last_peered":"2024-09-13T05:04:14.966129+0000","last_clean":"2024-09-13T05:04:14.966129+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:14.966129+0000","last_undegraded":"2024-09-13T05:04:14.966129+0000","last_fullsized":"2024-09-13T05:04:14.966129+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2874,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975235+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:15.975235+0000","last_peered":"2024-09-13T05:04:15.975235+0000","last_clean":"2024-09-13T05:04:15.975235+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:15.975235+0000","last_undegraded":"2024-09-13T05:04:15.975235+0000","last_fullsized":"2024-09-13T05:04:15.975235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4658,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118235+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.118235+0000","last_peered":"2024-09-13T05:04:16.118235+0000","last_clean":"2024-09-13T05:04:16.118235+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.118235+0000","last_undegraded":"2024-09-13T05:04:16.118235+0000","last_fullsized":"2024-09-13T05:04:16.118235+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2827,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966283+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:14.966283+0000","last_peered":"2024-09-13T05:04:14.966283+0000","last_clean":"2024-09-13T05:04:14.966283+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:14.966283+0000","last_undegraded":"2024-09-13T05:04:14.966283+0000","last_fullsized":"2024-09-13T05:04:14.966283+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3800,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965595+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:14.965595+0000","last_peered":"2024-09-13T05:04:14.965595+0000","last_clean":"2024-09-13T05:04:14.965595+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:14.965595+0000","last_undegraded":"2024-09-13T05:04:14.965595+0000","last_fullsized":"2024-09-13T05:04:14.965595+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5312,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:04:09.035348+0000","last_change":"2024-09-13T05:03:52.987943+0000","last_active":"2024-09-13T05:04:09.035348+0000","last_peered":"2024-09-13T05:04:09.035348+0000","last_clean":"2024-09-13T05:04:09.035348+0000","last_became_active":"2024-09-13T05:03:19.880315+0000","last_became_peered":"2024-09-13T05:03:19.880315+0000","last_unstale":"2024-09-13T05:04:09.035348+0000","last_undegraded":"2024-09-13T05:04:09.035348+0000","last_fullsized":"2024-09-13T05:04:09.035348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:15:27.243701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038146700000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5621,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170587+0000","last_change":"2024-09-13T05:03:52.986411+0000","last_active":"2024-09-13T05:04:10.170587+0000","last_peered":"2024-09-13T05:04:10.170587+0000","last_clean":"2024-09-13T05:04:10.170587+0000","last_became_active":"2024-09-13T05:03:19.880257+0000","last_became_peered":"2024-09-13T05:03:19.880257+0000","last_unstale":"2024-09-13T05:04:10.170587+0000","last_undegraded":"2024-09-13T05:04:10.170587+0000","last_fullsized":"2024-09-13T05:04:10.170587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:07.964691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3028,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972483+0000","last_change":"2024-09-13T05:04:15.972483+0000","last_active":"2024-09-13T05:04:15.972483+0000","last_peered":"2024-09-13T05:03:51.973455+0000","last_clean":"2024-09-13T05:03:51.973455+0000","last_became_active":"2024-09-13T05:03:39.927143+0000","last_became_peered":"2024-09-13T05:03:39.927143+0000","last_unstale":"2024-09-13T05:04:15.972483+0000","last_undegraded":"2024-09-13T05:04:15.972483+0000","last_fullsized":"2024-09-13T05:04:15.972483+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3101,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.972664+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:14.972664+0000","last_peered":"2024-09-13T05:04:14.972664+0000","last_clean":"2024-09-13T05:04:14.972664+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:14.972664+0000","last_undegraded":"2024-09-13T05:04:14.972664+0000","last_fullsized":"2024-09-13T05:04:14.972664+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4252,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170402+0000","last_change":"2024-09-13T05:03:52.986811+0000","last_active":"2024-09-13T05:04:10.170402+0000","last_peered":"2024-09-13T05:04:10.170402+0000","last_clean":"2024-09-13T05:04:10.170402+0000","last_became_active":"2024-09-13T05:03:19.880261+0000","last_became_peered":"2024-09-13T05:03:19.880261+0000","last_unstale":"2024-09-13T05:04:10.170402+0000","last_undegraded":"2024-09-13T05:04:10.170402+0000","last_fullsized":"2024-09-13T05:04:10.170402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:14.799250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966628+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:14.966628+0000","last_peered":"2024-09-13T05:04:14.966628+0000","last_clean":"2024-09-13T05:04:14.966628+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:14.966628+0000","last_undegraded":"2024-09-13T05:04:14.966628+0000","last_fullsized":"2024-09-13T05:04:14.966628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3279,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.773336+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.773336+0000","last_peered":"2024-09-13T05:04:15.773336+0000","last_clean":"2024-09-13T05:04:15.773336+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.773336+0000","last_undegraded":"2024-09-13T05:04:15.773336+0000","last_fullsized":"2024-09-13T05:04:15.773336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1617920,"data_stored":1335977,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40836,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331004,"statfs":{"total":96636764160,"available":96594948096,"internally_reserved":0,"allocated":1556480,"data_stored":745590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.435},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.5}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.625}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68752,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303088,"statfs":{"total":96636764160,"available":96566362112,"internally_reserved":0,"allocated":1572864,"data_stored":751451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":3.2400000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":2.681}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":3.3730000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.4380000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":1.242},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.37}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.35699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68748,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303092,"statfs":{"total":96636764160,"available":96566366208,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.22900000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.371},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.309},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.435}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.502}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.40699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343284,"statfs":{"total":96636764160,"available":96607522816,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.318}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.623}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72904,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298936,"statfs":{"total":96636764160,"available":96562110464,"internally_reserved":0,"allocated":1630208,"data_stored":1341819,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.318},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58900,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312940,"statfs":{"total":96636764160,"available":96576450560,"internally_reserved":0,"allocated":1572864,"data_stored":751451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.501},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.629},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:19.815 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-13T05:04:20.170 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:20.170 INFO:teuthology.orchestra.run.smithi053.stdout:{"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","health":{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg inactive, 3 pgs peering","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":155,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":132,"num_osds":8,"num_up_osds":8,"osd_up_since":1726203715,"num_in_osds":8,"osd_in_since":1726203839,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":36},{"state_name":"peering","count":5}],"num_pgs":41,"num_pools":3,"num_objects":343,"data_bytes":590406,"bytes_used":459096064,"bytes_avail":772635017216,"bytes_total":773094113280,"inactive_pgs_ratio":0.12195122241973877},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-13T05:02:57.949272+0000","services":{}},"progress_events":{}} 2024-09-13T05:04:20.183 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-13T05:04:20.183 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:20.468 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:20.469 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:20.480 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":197,"stamp":"2024-09-13T05:04:19.963325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448336,"kb_used_data":12360,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526384,"statfs":{"total":773094113280,"available":772635017216,"internally_reserved":0,"allocated":12656640,"data_stored":7180584,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001091"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2584,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966719+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:14.966719+0000","last_peered":"2024-09-13T05:04:14.966719+0000","last_clean":"2024-09-13T05:04:14.966719+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:14.966719+0000","last_undegraded":"2024-09-13T05:04:14.966719+0000","last_fullsized":"2024-09-13T05:04:14.966719+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2886,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966338+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:14.966338+0000","last_peered":"2024-09-13T05:04:14.966338+0000","last_clean":"2024-09-13T05:04:14.966338+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:14.966338+0000","last_undegraded":"2024-09-13T05:04:14.966338+0000","last_fullsized":"2024-09-13T05:04:14.966338+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2664,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965712+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:14.965712+0000","last_peered":"2024-09-13T05:04:14.965712+0000","last_clean":"2024-09-13T05:04:14.965712+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:14.965712+0000","last_undegraded":"2024-09-13T05:04:14.965712+0000","last_fullsized":"2024-09-13T05:04:14.965712+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5493,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965611+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:14.965611+0000","last_peered":"2024-09-13T05:04:14.965611+0000","last_clean":"2024-09-13T05:04:14.965611+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:14.965611+0000","last_undegraded":"2024-09-13T05:04:14.965611+0000","last_fullsized":"2024-09-13T05:04:14.965611+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5545,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966737+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:14.966737+0000","last_peered":"2024-09-13T05:04:14.966737+0000","last_clean":"2024-09-13T05:04:14.966737+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:14.966737+0000","last_undegraded":"2024-09-13T05:04:14.966737+0000","last_fullsized":"2024-09-13T05:04:14.966737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6375,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970825+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:14.970825+0000","last_peered":"2024-09-13T05:04:14.970825+0000","last_clean":"2024-09-13T05:04:14.970825+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:14.970825+0000","last_undegraded":"2024-09-13T05:04:14.970825+0000","last_fullsized":"2024-09-13T05:04:14.970825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2142,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772939+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.772939+0000","last_peered":"2024-09-13T05:04:15.772939+0000","last_clean":"2024-09-13T05:04:15.772939+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.772939+0000","last_undegraded":"2024-09-13T05:04:15.772939+0000","last_fullsized":"2024-09-13T05:04:15.772939+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2678,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966236+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:14.966236+0000","last_peered":"2024-09-13T05:04:14.966236+0000","last_clean":"2024-09-13T05:04:14.966236+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:14.966236+0000","last_undegraded":"2024-09-13T05:04:14.966236+0000","last_fullsized":"2024-09-13T05:04:14.966236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2793,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966774+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:14.966774+0000","last_peered":"2024-09-13T05:04:14.966774+0000","last_clean":"2024-09-13T05:04:14.966774+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:14.966774+0000","last_undegraded":"2024-09-13T05:04:14.966774+0000","last_fullsized":"2024-09-13T05:04:14.966774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3276,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965730+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:14.965730+0000","last_peered":"2024-09-13T05:04:14.965730+0000","last_clean":"2024-09-13T05:04:14.965730+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:14.965730+0000","last_undegraded":"2024-09-13T05:04:14.965730+0000","last_fullsized":"2024-09-13T05:04:14.965730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3681,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170347+0000","last_change":"2024-09-13T05:04:03.921625+0000","last_active":"2024-09-13T05:04:10.170347+0000","last_peered":"2024-09-13T05:04:10.170347+0000","last_clean":"2024-09-13T05:04:10.170347+0000","last_became_active":"2024-09-13T05:04:03.919391+0000","last_became_peered":"2024-09-13T05:04:03.919391+0000","last_unstale":"2024-09-13T05:04:10.170347+0000","last_undegraded":"2024-09-13T05:04:10.170347+0000","last_fullsized":"2024-09-13T05:04:10.170347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:09:04.622268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002304637,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2137,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970845+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:14.970845+0000","last_peered":"2024-09-13T05:04:14.970845+0000","last_clean":"2024-09-13T05:04:14.970845+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:14.970845+0000","last_undegraded":"2024-09-13T05:04:14.970845+0000","last_fullsized":"2024-09-13T05:04:14.970845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":86,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118273+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.118273+0000","last_peered":"2024-09-13T05:04:16.118273+0000","last_clean":"2024-09-13T05:04:16.118273+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.118273+0000","last_undegraded":"2024-09-13T05:04:16.118273+0000","last_fullsized":"2024-09-13T05:04:16.118273+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3183,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965786+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:14.965786+0000","last_peered":"2024-09-13T05:04:14.965786+0000","last_clean":"2024-09-13T05:04:14.965786+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:14.965786+0000","last_undegraded":"2024-09-13T05:04:14.965786+0000","last_fullsized":"2024-09-13T05:04:14.965786+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118261+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.118261+0000","last_peered":"2024-09-13T05:04:16.118261+0000","last_clean":"2024-09-13T05:04:16.118261+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.118261+0000","last_undegraded":"2024-09-13T05:04:16.118261+0000","last_fullsized":"2024-09-13T05:04:16.118261+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4990,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772993+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.772993+0000","last_peered":"2024-09-13T05:04:15.772993+0000","last_clean":"2024-09-13T05:04:15.772993+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.772993+0000","last_undegraded":"2024-09-13T05:04:15.772993+0000","last_fullsized":"2024-09-13T05:04:15.772993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":131,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.970912+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:14.970912+0000","last_peered":"2024-09-13T05:04:14.970912+0000","last_clean":"2024-09-13T05:04:14.970912+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:14.970912+0000","last_undegraded":"2024-09-13T05:04:14.970912+0000","last_fullsized":"2024-09-13T05:04:14.970912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2550,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.967026+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:14.967026+0000","last_peered":"2024-09-13T05:04:14.967026+0000","last_clean":"2024-09-13T05:04:14.967026+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:14.967026+0000","last_undegraded":"2024-09-13T05:04:14.967026+0000","last_fullsized":"2024-09-13T05:04:14.967026+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966392+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:14.966392+0000","last_peered":"2024-09-13T05:04:14.966392+0000","last_clean":"2024-09-13T05:04:14.966392+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:14.966392+0000","last_undegraded":"2024-09-13T05:04:14.966392+0000","last_fullsized":"2024-09-13T05:04:14.966392+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3463,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974508+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:15.974508+0000","last_peered":"2024-09-13T05:04:15.974508+0000","last_clean":"2024-09-13T05:04:15.974508+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:15.974508+0000","last_undegraded":"2024-09-13T05:04:15.974508+0000","last_fullsized":"2024-09-13T05:04:15.974508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2588,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966269+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:14.966269+0000","last_peered":"2024-09-13T05:04:14.966269+0000","last_clean":"2024-09-13T05:04:14.966269+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:14.966269+0000","last_undegraded":"2024-09-13T05:04:14.966269+0000","last_fullsized":"2024-09-13T05:04:14.966269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":220,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.971997+0000","last_change":"2024-09-13T05:04:15.971997+0000","last_active":"2024-09-13T05:04:15.971997+0000","last_peered":"2024-09-13T05:03:17.966011+0000","last_clean":"2024-09-13T05:03:17.966011+0000","last_became_active":"2024-09-13T05:01:57.174886+0000","last_became_peered":"2024-09-13T05:01:57.174886+0000","last_unstale":"2024-09-13T05:04:15.971997+0000","last_undegraded":"2024-09-13T05:04:15.971997+0000","last_fullsized":"2024-09-13T05:04:15.971997+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":132,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772940+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.772940+0000","last_peered":"2024-09-13T05:04:15.772940+0000","last_clean":"2024-09-13T05:04:15.772940+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.772940+0000","last_undegraded":"2024-09-13T05:04:15.772940+0000","last_fullsized":"2024-09-13T05:04:15.772940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4224,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.772922+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.772922+0000","last_peered":"2024-09-13T05:04:15.772922+0000","last_clean":"2024-09-13T05:04:15.772922+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.772922+0000","last_undegraded":"2024-09-13T05:04:15.772922+0000","last_fullsized":"2024-09-13T05:04:15.772922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":77,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972139+0000","last_change":"2024-09-13T05:04:15.972139+0000","last_active":"2024-09-13T05:04:15.972139+0000","last_peered":"2024-09-13T05:03:17.966055+0000","last_clean":"2024-09-13T05:03:17.966055+0000","last_became_active":"2024-09-13T05:01:57.172837+0000","last_became_peered":"2024-09-13T05:01:57.172837+0000","last_unstale":"2024-09-13T05:04:15.972139+0000","last_undegraded":"2024-09-13T05:04:15.972139+0000","last_fullsized":"2024-09-13T05:04:15.972139+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3692,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972152+0000","last_change":"2024-09-13T05:04:15.972152+0000","last_active":"2024-09-13T05:04:15.972152+0000","last_peered":"2024-09-13T05:03:53.037707+0000","last_clean":"2024-09-13T05:03:53.037707+0000","last_became_active":"2024-09-13T05:03:51.976212+0000","last_became_peered":"2024-09-13T05:03:51.976212+0000","last_unstale":"2024-09-13T05:04:15.972152+0000","last_undegraded":"2024-09-13T05:04:15.972152+0000","last_fullsized":"2024-09-13T05:04:15.972152+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":130,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965596+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:14.965596+0000","last_peered":"2024-09-13T05:04:14.965596+0000","last_clean":"2024-09-13T05:04:14.965596+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:14.965596+0000","last_undegraded":"2024-09-13T05:04:14.965596+0000","last_fullsized":"2024-09-13T05:04:14.965596+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3912,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966471+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:14.966471+0000","last_peered":"2024-09-13T05:04:14.966471+0000","last_clean":"2024-09-13T05:04:14.966471+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:14.966471+0000","last_undegraded":"2024-09-13T05:04:14.966471+0000","last_fullsized":"2024-09-13T05:04:14.966471+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972960+0000","last_change":"2024-09-13T05:03:18.975073+0000","last_active":"2024-09-13T05:01:56.162205+0000","last_peered":"2024-09-13T05:01:56.162205+0000","last_clean":"2024-09-13T05:01:56.162205+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-13T05:04:15.972960+0000","last_undegraded":"2024-09-13T05:04:15.972960+0000","last_fullsized":"2024-09-13T05:04:15.972960+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3732,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966129+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:14.966129+0000","last_peered":"2024-09-13T05:04:14.966129+0000","last_clean":"2024-09-13T05:04:14.966129+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:14.966129+0000","last_undegraded":"2024-09-13T05:04:14.966129+0000","last_fullsized":"2024-09-13T05:04:14.966129+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2874,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975235+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:15.975235+0000","last_peered":"2024-09-13T05:04:15.975235+0000","last_clean":"2024-09-13T05:04:15.975235+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:15.975235+0000","last_undegraded":"2024-09-13T05:04:15.975235+0000","last_fullsized":"2024-09-13T05:04:15.975235+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4658,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118235+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.118235+0000","last_peered":"2024-09-13T05:04:16.118235+0000","last_clean":"2024-09-13T05:04:16.118235+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.118235+0000","last_undegraded":"2024-09-13T05:04:16.118235+0000","last_fullsized":"2024-09-13T05:04:16.118235+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2827,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966283+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:14.966283+0000","last_peered":"2024-09-13T05:04:14.966283+0000","last_clean":"2024-09-13T05:04:14.966283+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:14.966283+0000","last_undegraded":"2024-09-13T05:04:14.966283+0000","last_fullsized":"2024-09-13T05:04:14.966283+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3800,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.965595+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:14.965595+0000","last_peered":"2024-09-13T05:04:14.965595+0000","last_clean":"2024-09-13T05:04:14.965595+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:14.965595+0000","last_undegraded":"2024-09-13T05:04:14.965595+0000","last_fullsized":"2024-09-13T05:04:14.965595+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5312,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-09-13T05:04:09.035348+0000","last_change":"2024-09-13T05:03:52.987943+0000","last_active":"2024-09-13T05:04:09.035348+0000","last_peered":"2024-09-13T05:04:09.035348+0000","last_clean":"2024-09-13T05:04:09.035348+0000","last_became_active":"2024-09-13T05:03:19.880315+0000","last_became_peered":"2024-09-13T05:03:19.880315+0000","last_unstale":"2024-09-13T05:04:09.035348+0000","last_undegraded":"2024-09-13T05:04:09.035348+0000","last_fullsized":"2024-09-13T05:04:09.035348+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:15:27.243701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038146700000000003,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5621,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170587+0000","last_change":"2024-09-13T05:03:52.986411+0000","last_active":"2024-09-13T05:04:10.170587+0000","last_peered":"2024-09-13T05:04:10.170587+0000","last_clean":"2024-09-13T05:04:10.170587+0000","last_became_active":"2024-09-13T05:03:19.880257+0000","last_became_peered":"2024-09-13T05:03:19.880257+0000","last_unstale":"2024-09-13T05:04:10.170587+0000","last_undegraded":"2024-09-13T05:04:10.170587+0000","last_fullsized":"2024-09-13T05:04:10.170587+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:13:07.964691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3028,"reported_epoch":131,"state":"peering","last_fresh":"2024-09-13T05:04:15.972483+0000","last_change":"2024-09-13T05:04:15.972483+0000","last_active":"2024-09-13T05:04:15.972483+0000","last_peered":"2024-09-13T05:03:51.973455+0000","last_clean":"2024-09-13T05:03:51.973455+0000","last_became_active":"2024-09-13T05:03:39.927143+0000","last_became_peered":"2024-09-13T05:03:39.927143+0000","last_unstale":"2024-09-13T05:04:15.972483+0000","last_undegraded":"2024-09-13T05:04:15.972483+0000","last_fullsized":"2024-09-13T05:04:15.972483+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3101,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.972664+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:14.972664+0000","last_peered":"2024-09-13T05:04:14.972664+0000","last_clean":"2024-09-13T05:04:14.972664+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:14.972664+0000","last_undegraded":"2024-09-13T05:04:14.972664+0000","last_fullsized":"2024-09-13T05:04:14.972664+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4252,"reported_epoch":125,"state":"active+clean","last_fresh":"2024-09-13T05:04:10.170402+0000","last_change":"2024-09-13T05:03:52.986811+0000","last_active":"2024-09-13T05:04:10.170402+0000","last_peered":"2024-09-13T05:04:10.170402+0000","last_clean":"2024-09-13T05:04:10.170402+0000","last_became_active":"2024-09-13T05:03:19.880261+0000","last_became_peered":"2024-09-13T05:03:19.880261+0000","last_unstale":"2024-09-13T05:04:10.170402+0000","last_undegraded":"2024-09-13T05:04:10.170402+0000","last_fullsized":"2024-09-13T05:04:10.170402+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:50:14.799250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3844,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:14.966628+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:14.966628+0000","last_peered":"2024-09-13T05:04:14.966628+0000","last_clean":"2024-09-13T05:04:14.966628+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:14.966628+0000","last_undegraded":"2024-09-13T05:04:14.966628+0000","last_fullsized":"2024-09-13T05:04:14.966628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3279,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.773336+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.773336+0000","last_peered":"2024-09-13T05:04:15.773336+0000","last_clean":"2024-09-13T05:04:15.773336+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.773336+0000","last_undegraded":"2024-09-13T05:04:15.773336+0000","last_fullsized":"2024-09-13T05:04:15.773336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2892,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58936,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312904,"statfs":{"total":96636764160,"available":96576413696,"internally_reserved":0,"allocated":1617920,"data_stored":1335977,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40836,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94331004,"statfs":{"total":96636764160,"available":96594948096,"internally_reserved":0,"allocated":1556480,"data_stored":745590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.61199999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.435},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.5}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50704,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321136,"statfs":{"total":96636764160,"available":96584843264,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.625}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.72399999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.58399999999999996}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.44500000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.49399999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.47799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68752,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303088,"statfs":{"total":96636764160,"available":96566362112,"internally_reserved":0,"allocated":1572864,"data_stored":751451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":3.2400000000000002},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":2.681}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":3.3730000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":1.0700000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":3.4380000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":1.3129999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":1.242},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.42999999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.37}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.46100000000000002}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.35699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68748,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303092,"statfs":{"total":96636764160,"available":96566366208,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.22900000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.371},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.26500000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.309},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.435}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.46899999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.502}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.45000000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.40699999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28556,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343284,"statfs":{"total":96636764160,"available":96607522816,"internally_reserved":0,"allocated":1568768,"data_stored":751432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.318}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.623}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.56299999999999994}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.53500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72904,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298936,"statfs":{"total":96636764160,"available":96562110464,"internally_reserved":0,"allocated":1630208,"data_stored":1341819,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.23999999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.318},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.33700000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.60899999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66200000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58900,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312940,"statfs":{"total":96636764160,"available":96576450560,"internally_reserved":0,"allocated":1572864,"data_stored":751451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.501},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.40200000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.629},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.57999999999999996}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:23.484 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:23.789 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:23.789 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:23.800 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-13T05:04:21.963926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448456,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526264,"statfs":{"total":773094113280,"available":772634894336,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.999644"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2585,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974720+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:15.974720+0000","last_peered":"2024-09-13T05:04:15.974720+0000","last_clean":"2024-09-13T05:04:15.974720+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:15.974720+0000","last_undegraded":"2024-09-13T05:04:15.974720+0000","last_fullsized":"2024-09-13T05:04:15.974720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2887,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975359+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:15.975359+0000","last_peered":"2024-09-13T05:04:15.975359+0000","last_clean":"2024-09-13T05:04:15.975359+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:15.975359+0000","last_undegraded":"2024-09-13T05:04:15.975359+0000","last_fullsized":"2024-09-13T05:04:15.975359+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2665,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977022+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:15.977022+0000","last_peered":"2024-09-13T05:04:15.977022+0000","last_clean":"2024-09-13T05:04:15.977022+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:15.977022+0000","last_undegraded":"2024-09-13T05:04:15.977022+0000","last_fullsized":"2024-09-13T05:04:15.977022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5494,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976175+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:15.976175+0000","last_peered":"2024-09-13T05:04:15.976175+0000","last_clean":"2024-09-13T05:04:15.976175+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:15.976175+0000","last_undegraded":"2024-09-13T05:04:15.976175+0000","last_fullsized":"2024-09-13T05:04:15.976175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5546,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974869+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:15.974869+0000","last_peered":"2024-09-13T05:04:15.974869+0000","last_clean":"2024-09-13T05:04:15.974869+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:15.974869+0000","last_undegraded":"2024-09-13T05:04:15.974869+0000","last_fullsized":"2024-09-13T05:04:15.974869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974392+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:15.974392+0000","last_peered":"2024-09-13T05:04:15.974392+0000","last_clean":"2024-09-13T05:04:15.974392+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:15.974392+0000","last_undegraded":"2024-09-13T05:04:15.974392+0000","last_fullsized":"2024-09-13T05:04:15.974392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2143,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971355+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.971355+0000","last_peered":"2024-09-13T05:04:15.971355+0000","last_clean":"2024-09-13T05:04:15.971355+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.971355+0000","last_undegraded":"2024-09-13T05:04:15.971355+0000","last_fullsized":"2024-09-13T05:04:15.971355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2679,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975612+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:15.975612+0000","last_peered":"2024-09-13T05:04:15.975612+0000","last_clean":"2024-09-13T05:04:15.975612+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:15.975612+0000","last_undegraded":"2024-09-13T05:04:15.975612+0000","last_fullsized":"2024-09-13T05:04:15.975612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2794,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974749+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:15.974749+0000","last_peered":"2024-09-13T05:04:15.974749+0000","last_clean":"2024-09-13T05:04:15.974749+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:15.974749+0000","last_undegraded":"2024-09-13T05:04:15.974749+0000","last_fullsized":"2024-09-13T05:04:15.974749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3277,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976313+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:15.976313+0000","last_peered":"2024-09-13T05:04:15.976313+0000","last_clean":"2024-09-13T05:04:15.976313+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:15.976313+0000","last_undegraded":"2024-09-13T05:04:15.976313+0000","last_fullsized":"2024-09-13T05:04:15.976313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.990775+0000","last_change":"2024-09-13T05:04:16.990775+0000","last_active":"2024-09-13T05:04:16.990775+0000","last_peered":"2024-09-13T05:04:16.990775+0000","last_clean":"2024-09-13T05:04:16.990775+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:04:16.990775+0000","last_undegraded":"2024-09-13T05:04:16.990775+0000","last_fullsized":"2024-09-13T05:04:16.990775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:52:14.353780+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2138,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974417+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:15.974417+0000","last_peered":"2024-09-13T05:04:15.974417+0000","last_clean":"2024-09-13T05:04:15.974417+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:15.974417+0000","last_undegraded":"2024-09-13T05:04:15.974417+0000","last_fullsized":"2024-09-13T05:04:15.974417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3184,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977122+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:15.977122+0000","last_peered":"2024-09-13T05:04:15.977122+0000","last_clean":"2024-09-13T05:04:15.977122+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:15.977122+0000","last_undegraded":"2024-09-13T05:04:15.977122+0000","last_fullsized":"2024-09-13T05:04:15.977122+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4991,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971444+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.971444+0000","last_peered":"2024-09-13T05:04:15.971444+0000","last_clean":"2024-09-13T05:04:15.971444+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.971444+0000","last_undegraded":"2024-09-13T05:04:15.971444+0000","last_fullsized":"2024-09-13T05:04:15.971444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974723+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:15.974723+0000","last_peered":"2024-09-13T05:04:15.974723+0000","last_clean":"2024-09-13T05:04:15.974723+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:15.974723+0000","last_undegraded":"2024-09-13T05:04:15.974723+0000","last_fullsized":"2024-09-13T05:04:15.974723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2551,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976149+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:15.976149+0000","last_peered":"2024-09-13T05:04:15.976149+0000","last_clean":"2024-09-13T05:04:15.976149+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:15.976149+0000","last_undegraded":"2024-09-13T05:04:15.976149+0000","last_fullsized":"2024-09-13T05:04:15.976149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975969+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:15.975969+0000","last_peered":"2024-09-13T05:04:15.975969+0000","last_clean":"2024-09-13T05:04:15.975969+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:15.975969+0000","last_undegraded":"2024-09-13T05:04:15.975969+0000","last_fullsized":"2024-09-13T05:04:15.975969+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3464,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.122024+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:16.122024+0000","last_peered":"2024-09-13T05:04:16.122024+0000","last_clean":"2024-09-13T05:04:16.122024+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:16.122024+0000","last_undegraded":"2024-09-13T05:04:16.122024+0000","last_fullsized":"2024-09-13T05:04:16.122024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2589,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974247+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:15.974247+0000","last_peered":"2024-09-13T05:04:15.974247+0000","last_clean":"2024-09-13T05:04:15.974247+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:15.974247+0000","last_undegraded":"2024-09-13T05:04:15.974247+0000","last_fullsized":"2024-09-13T05:04:15.974247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":228,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986088+0000","last_change":"2024-09-13T05:04:16.986088+0000","last_active":"2024-09-13T05:04:16.986088+0000","last_peered":"2024-09-13T05:04:16.986088+0000","last_clean":"2024-09-13T05:04:16.986088+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:16.986088+0000","last_undegraded":"2024-09-13T05:04:16.986088+0000","last_fullsized":"2024-09-13T05:04:16.986088+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":133,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971382+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.971382+0000","last_peered":"2024-09-13T05:04:15.971382+0000","last_clean":"2024-09-13T05:04:15.971382+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.971382+0000","last_undegraded":"2024-09-13T05:04:15.971382+0000","last_fullsized":"2024-09-13T05:04:15.971382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4225,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971385+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.971385+0000","last_peered":"2024-09-13T05:04:15.971385+0000","last_clean":"2024-09-13T05:04:15.971385+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.971385+0000","last_undegraded":"2024-09-13T05:04:15.971385+0000","last_fullsized":"2024-09-13T05:04:15.971385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":85,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.988998+0000","last_change":"2024-09-13T05:04:16.988998+0000","last_active":"2024-09-13T05:04:16.988998+0000","last_peered":"2024-09-13T05:04:16.988998+0000","last_clean":"2024-09-13T05:04:16.988998+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:16.988998+0000","last_undegraded":"2024-09-13T05:04:16.988998+0000","last_fullsized":"2024-09-13T05:04:16.988998+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.991471+0000","last_change":"2024-09-13T05:04:16.991471+0000","last_active":"2024-09-13T05:04:16.991471+0000","last_peered":"2024-09-13T05:04:16.991471+0000","last_clean":"2024-09-13T05:04:16.991471+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:16.991471+0000","last_undegraded":"2024-09-13T05:04:16.991471+0000","last_fullsized":"2024-09-13T05:04:16.991471+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976066+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:15.976066+0000","last_peered":"2024-09-13T05:04:15.976066+0000","last_clean":"2024-09-13T05:04:15.976066+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:15.976066+0000","last_undegraded":"2024-09-13T05:04:15.976066+0000","last_fullsized":"2024-09-13T05:04:15.976066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3913,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975702+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:15.975702+0000","last_peered":"2024-09-13T05:04:15.975702+0000","last_clean":"2024-09-13T05:04:15.975702+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:15.975702+0000","last_undegraded":"2024-09-13T05:04:15.975702+0000","last_fullsized":"2024-09-13T05:04:15.975702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.989101+0000","last_change":"2024-09-13T05:04:16.989101+0000","last_active":"2024-09-13T05:04:16.989101+0000","last_peered":"2024-09-13T05:04:16.989101+0000","last_clean":"2024-09-13T05:04:16.989101+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:16.989101+0000","last_undegraded":"2024-09-13T05:04:16.989101+0000","last_fullsized":"2024-09-13T05:04:16.989101+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3733,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975603+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:15.975603+0000","last_peered":"2024-09-13T05:04:15.975603+0000","last_clean":"2024-09-13T05:04:15.975603+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:15.975603+0000","last_undegraded":"2024-09-13T05:04:15.975603+0000","last_fullsized":"2024-09-13T05:04:15.975603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2875,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118779+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:16.118779+0000","last_peered":"2024-09-13T05:04:16.118779+0000","last_clean":"2024-09-13T05:04:16.118779+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:16.118779+0000","last_undegraded":"2024-09-13T05:04:16.118779+0000","last_fullsized":"2024-09-13T05:04:16.118779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4659,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348167+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.348167+0000","last_peered":"2024-09-13T05:04:16.348167+0000","last_clean":"2024-09-13T05:04:16.348167+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.348167+0000","last_undegraded":"2024-09-13T05:04:16.348167+0000","last_fullsized":"2024-09-13T05:04:16.348167+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2828,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975851+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:15.975851+0000","last_peered":"2024-09-13T05:04:15.975851+0000","last_clean":"2024-09-13T05:04:15.975851+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:15.975851+0000","last_undegraded":"2024-09-13T05:04:15.975851+0000","last_fullsized":"2024-09-13T05:04:15.975851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3802,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982535+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:16.982535+0000","last_peered":"2024-09-13T05:04:16.982535+0000","last_clean":"2024-09-13T05:04:16.982535+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:16.982535+0000","last_undegraded":"2024-09-13T05:04:16.982535+0000","last_fullsized":"2024-09-13T05:04:16.982535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5292,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986129+0000","last_change":"2024-09-13T05:04:16.985647+0000","last_active":"2024-09-13T05:04:16.986129+0000","last_peered":"2024-09-13T05:04:16.986129+0000","last_clean":"2024-09-13T05:04:16.986129+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:04:16.986129+0000","last_undegraded":"2024-09-13T05:04:16.986129+0000","last_fullsized":"2024-09-13T05:04:16.986129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:39.134756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5611,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984699+0000","last_change":"2024-09-13T05:04:16.984083+0000","last_active":"2024-09-13T05:04:16.984699+0000","last_peered":"2024-09-13T05:04:16.984699+0000","last_clean":"2024-09-13T05:04:16.984699+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:16.984699+0000","last_undegraded":"2024-09-13T05:04:16.984699+0000","last_fullsized":"2024-09-13T05:04:16.984699+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:51.096635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3036,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984387+0000","last_change":"2024-09-13T05:04:16.984387+0000","last_active":"2024-09-13T05:04:16.984387+0000","last_peered":"2024-09-13T05:04:16.984387+0000","last_clean":"2024-09-13T05:04:16.984387+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:16.984387+0000","last_undegraded":"2024-09-13T05:04:16.984387+0000","last_fullsized":"2024-09-13T05:04:16.984387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3102,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348129+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:16.348129+0000","last_peered":"2024-09-13T05:04:16.348129+0000","last_clean":"2024-09-13T05:04:16.348129+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:16.348129+0000","last_undegraded":"2024-09-13T05:04:16.348129+0000","last_fullsized":"2024-09-13T05:04:16.348129+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982881+0000","last_change":"2024-09-13T05:04:16.982881+0000","last_active":"2024-09-13T05:04:16.982881+0000","last_peered":"2024-09-13T05:04:16.982881+0000","last_clean":"2024-09-13T05:04:16.982881+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:16.982881+0000","last_undegraded":"2024-09-13T05:04:16.982881+0000","last_fullsized":"2024-09-13T05:04:16.982881+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:46.387386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3845,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974449+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:15.974449+0000","last_peered":"2024-09-13T05:04:15.974449+0000","last_clean":"2024-09-13T05:04:15.974449+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:15.974449+0000","last_undegraded":"2024-09-13T05:04:15.974449+0000","last_fullsized":"2024-09-13T05:04:15.974449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3280,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971480+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.971480+0000","last_peered":"2024-09-13T05:04:15.971480+0000","last_clean":"2024-09-13T05:04:15.971480+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.971480+0000","last_undegraded":"2024-09-13T05:04:15.971480+0000","last_fullsized":"2024-09-13T05:04:15.971480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.622},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50708,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321132,"statfs":{"total":96636764160,"available":96584839168,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.498},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68768,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303072,"statfs":{"total":96636764160,"available":96566345728,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.496},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.746},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68764,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303076,"statfs":{"total":96636764160,"available":96566349824,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.378},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.29099999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.371},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.30299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.496},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.371},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":1.6539999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":1.6859999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":1.768}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58904,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312936,"statfs":{"total":96636764160,"available":96576446464,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.317},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.30199999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:23.803 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-13T05:04:23.804 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-13T05:04:24.108 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-13T05:04:24.117 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-13T05:04:24.117 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:24.401 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:24.401 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:24.412 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-13T05:04:23.964478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448456,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526264,"statfs":{"total":773094113280,"available":772634894336,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001840"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2585,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974720+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:15.974720+0000","last_peered":"2024-09-13T05:04:15.974720+0000","last_clean":"2024-09-13T05:04:15.974720+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:15.974720+0000","last_undegraded":"2024-09-13T05:04:15.974720+0000","last_fullsized":"2024-09-13T05:04:15.974720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2887,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975359+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:15.975359+0000","last_peered":"2024-09-13T05:04:15.975359+0000","last_clean":"2024-09-13T05:04:15.975359+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:15.975359+0000","last_undegraded":"2024-09-13T05:04:15.975359+0000","last_fullsized":"2024-09-13T05:04:15.975359+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2665,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977022+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:15.977022+0000","last_peered":"2024-09-13T05:04:15.977022+0000","last_clean":"2024-09-13T05:04:15.977022+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:15.977022+0000","last_undegraded":"2024-09-13T05:04:15.977022+0000","last_fullsized":"2024-09-13T05:04:15.977022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5494,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976175+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:15.976175+0000","last_peered":"2024-09-13T05:04:15.976175+0000","last_clean":"2024-09-13T05:04:15.976175+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:15.976175+0000","last_undegraded":"2024-09-13T05:04:15.976175+0000","last_fullsized":"2024-09-13T05:04:15.976175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5546,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974869+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:15.974869+0000","last_peered":"2024-09-13T05:04:15.974869+0000","last_clean":"2024-09-13T05:04:15.974869+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:15.974869+0000","last_undegraded":"2024-09-13T05:04:15.974869+0000","last_fullsized":"2024-09-13T05:04:15.974869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974392+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:15.974392+0000","last_peered":"2024-09-13T05:04:15.974392+0000","last_clean":"2024-09-13T05:04:15.974392+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:15.974392+0000","last_undegraded":"2024-09-13T05:04:15.974392+0000","last_fullsized":"2024-09-13T05:04:15.974392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2143,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971355+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.971355+0000","last_peered":"2024-09-13T05:04:15.971355+0000","last_clean":"2024-09-13T05:04:15.971355+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.971355+0000","last_undegraded":"2024-09-13T05:04:15.971355+0000","last_fullsized":"2024-09-13T05:04:15.971355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2679,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975612+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:15.975612+0000","last_peered":"2024-09-13T05:04:15.975612+0000","last_clean":"2024-09-13T05:04:15.975612+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:15.975612+0000","last_undegraded":"2024-09-13T05:04:15.975612+0000","last_fullsized":"2024-09-13T05:04:15.975612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2794,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974749+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:15.974749+0000","last_peered":"2024-09-13T05:04:15.974749+0000","last_clean":"2024-09-13T05:04:15.974749+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:15.974749+0000","last_undegraded":"2024-09-13T05:04:15.974749+0000","last_fullsized":"2024-09-13T05:04:15.974749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3277,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976313+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:15.976313+0000","last_peered":"2024-09-13T05:04:15.976313+0000","last_clean":"2024-09-13T05:04:15.976313+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:15.976313+0000","last_undegraded":"2024-09-13T05:04:15.976313+0000","last_fullsized":"2024-09-13T05:04:15.976313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.990775+0000","last_change":"2024-09-13T05:04:16.990775+0000","last_active":"2024-09-13T05:04:16.990775+0000","last_peered":"2024-09-13T05:04:16.990775+0000","last_clean":"2024-09-13T05:04:16.990775+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:04:16.990775+0000","last_undegraded":"2024-09-13T05:04:16.990775+0000","last_fullsized":"2024-09-13T05:04:16.990775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:52:14.353780+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2138,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974417+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:15.974417+0000","last_peered":"2024-09-13T05:04:15.974417+0000","last_clean":"2024-09-13T05:04:15.974417+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:15.974417+0000","last_undegraded":"2024-09-13T05:04:15.974417+0000","last_fullsized":"2024-09-13T05:04:15.974417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3184,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977122+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:15.977122+0000","last_peered":"2024-09-13T05:04:15.977122+0000","last_clean":"2024-09-13T05:04:15.977122+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:15.977122+0000","last_undegraded":"2024-09-13T05:04:15.977122+0000","last_fullsized":"2024-09-13T05:04:15.977122+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4991,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971444+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.971444+0000","last_peered":"2024-09-13T05:04:15.971444+0000","last_clean":"2024-09-13T05:04:15.971444+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.971444+0000","last_undegraded":"2024-09-13T05:04:15.971444+0000","last_fullsized":"2024-09-13T05:04:15.971444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974723+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:15.974723+0000","last_peered":"2024-09-13T05:04:15.974723+0000","last_clean":"2024-09-13T05:04:15.974723+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:15.974723+0000","last_undegraded":"2024-09-13T05:04:15.974723+0000","last_fullsized":"2024-09-13T05:04:15.974723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2551,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976149+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:15.976149+0000","last_peered":"2024-09-13T05:04:15.976149+0000","last_clean":"2024-09-13T05:04:15.976149+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:15.976149+0000","last_undegraded":"2024-09-13T05:04:15.976149+0000","last_fullsized":"2024-09-13T05:04:15.976149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975969+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:15.975969+0000","last_peered":"2024-09-13T05:04:15.975969+0000","last_clean":"2024-09-13T05:04:15.975969+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:15.975969+0000","last_undegraded":"2024-09-13T05:04:15.975969+0000","last_fullsized":"2024-09-13T05:04:15.975969+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3464,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.122024+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:16.122024+0000","last_peered":"2024-09-13T05:04:16.122024+0000","last_clean":"2024-09-13T05:04:16.122024+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:16.122024+0000","last_undegraded":"2024-09-13T05:04:16.122024+0000","last_fullsized":"2024-09-13T05:04:16.122024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2589,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974247+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:15.974247+0000","last_peered":"2024-09-13T05:04:15.974247+0000","last_clean":"2024-09-13T05:04:15.974247+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:15.974247+0000","last_undegraded":"2024-09-13T05:04:15.974247+0000","last_fullsized":"2024-09-13T05:04:15.974247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":228,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986088+0000","last_change":"2024-09-13T05:04:16.986088+0000","last_active":"2024-09-13T05:04:16.986088+0000","last_peered":"2024-09-13T05:04:16.986088+0000","last_clean":"2024-09-13T05:04:16.986088+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:16.986088+0000","last_undegraded":"2024-09-13T05:04:16.986088+0000","last_fullsized":"2024-09-13T05:04:16.986088+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":133,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971382+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.971382+0000","last_peered":"2024-09-13T05:04:15.971382+0000","last_clean":"2024-09-13T05:04:15.971382+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.971382+0000","last_undegraded":"2024-09-13T05:04:15.971382+0000","last_fullsized":"2024-09-13T05:04:15.971382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4225,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971385+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.971385+0000","last_peered":"2024-09-13T05:04:15.971385+0000","last_clean":"2024-09-13T05:04:15.971385+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.971385+0000","last_undegraded":"2024-09-13T05:04:15.971385+0000","last_fullsized":"2024-09-13T05:04:15.971385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":85,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.988998+0000","last_change":"2024-09-13T05:04:16.988998+0000","last_active":"2024-09-13T05:04:16.988998+0000","last_peered":"2024-09-13T05:04:16.988998+0000","last_clean":"2024-09-13T05:04:16.988998+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:16.988998+0000","last_undegraded":"2024-09-13T05:04:16.988998+0000","last_fullsized":"2024-09-13T05:04:16.988998+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.991471+0000","last_change":"2024-09-13T05:04:16.991471+0000","last_active":"2024-09-13T05:04:16.991471+0000","last_peered":"2024-09-13T05:04:16.991471+0000","last_clean":"2024-09-13T05:04:16.991471+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:16.991471+0000","last_undegraded":"2024-09-13T05:04:16.991471+0000","last_fullsized":"2024-09-13T05:04:16.991471+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976066+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:15.976066+0000","last_peered":"2024-09-13T05:04:15.976066+0000","last_clean":"2024-09-13T05:04:15.976066+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:15.976066+0000","last_undegraded":"2024-09-13T05:04:15.976066+0000","last_fullsized":"2024-09-13T05:04:15.976066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3913,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975702+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:15.975702+0000","last_peered":"2024-09-13T05:04:15.975702+0000","last_clean":"2024-09-13T05:04:15.975702+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:15.975702+0000","last_undegraded":"2024-09-13T05:04:15.975702+0000","last_fullsized":"2024-09-13T05:04:15.975702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.989101+0000","last_change":"2024-09-13T05:04:16.989101+0000","last_active":"2024-09-13T05:04:16.989101+0000","last_peered":"2024-09-13T05:04:16.989101+0000","last_clean":"2024-09-13T05:04:16.989101+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:16.989101+0000","last_undegraded":"2024-09-13T05:04:16.989101+0000","last_fullsized":"2024-09-13T05:04:16.989101+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3733,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975603+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:15.975603+0000","last_peered":"2024-09-13T05:04:15.975603+0000","last_clean":"2024-09-13T05:04:15.975603+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:15.975603+0000","last_undegraded":"2024-09-13T05:04:15.975603+0000","last_fullsized":"2024-09-13T05:04:15.975603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2875,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118779+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:16.118779+0000","last_peered":"2024-09-13T05:04:16.118779+0000","last_clean":"2024-09-13T05:04:16.118779+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:16.118779+0000","last_undegraded":"2024-09-13T05:04:16.118779+0000","last_fullsized":"2024-09-13T05:04:16.118779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4659,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348167+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.348167+0000","last_peered":"2024-09-13T05:04:16.348167+0000","last_clean":"2024-09-13T05:04:16.348167+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.348167+0000","last_undegraded":"2024-09-13T05:04:16.348167+0000","last_fullsized":"2024-09-13T05:04:16.348167+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2828,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975851+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:15.975851+0000","last_peered":"2024-09-13T05:04:15.975851+0000","last_clean":"2024-09-13T05:04:15.975851+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:15.975851+0000","last_undegraded":"2024-09-13T05:04:15.975851+0000","last_fullsized":"2024-09-13T05:04:15.975851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3802,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982535+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:16.982535+0000","last_peered":"2024-09-13T05:04:16.982535+0000","last_clean":"2024-09-13T05:04:16.982535+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:16.982535+0000","last_undegraded":"2024-09-13T05:04:16.982535+0000","last_fullsized":"2024-09-13T05:04:16.982535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5292,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986129+0000","last_change":"2024-09-13T05:04:16.985647+0000","last_active":"2024-09-13T05:04:16.986129+0000","last_peered":"2024-09-13T05:04:16.986129+0000","last_clean":"2024-09-13T05:04:16.986129+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:04:16.986129+0000","last_undegraded":"2024-09-13T05:04:16.986129+0000","last_fullsized":"2024-09-13T05:04:16.986129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:39.134756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5611,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984699+0000","last_change":"2024-09-13T05:04:16.984083+0000","last_active":"2024-09-13T05:04:16.984699+0000","last_peered":"2024-09-13T05:04:16.984699+0000","last_clean":"2024-09-13T05:04:16.984699+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:16.984699+0000","last_undegraded":"2024-09-13T05:04:16.984699+0000","last_fullsized":"2024-09-13T05:04:16.984699+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:51.096635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3036,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984387+0000","last_change":"2024-09-13T05:04:16.984387+0000","last_active":"2024-09-13T05:04:16.984387+0000","last_peered":"2024-09-13T05:04:16.984387+0000","last_clean":"2024-09-13T05:04:16.984387+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:16.984387+0000","last_undegraded":"2024-09-13T05:04:16.984387+0000","last_fullsized":"2024-09-13T05:04:16.984387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3102,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348129+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:16.348129+0000","last_peered":"2024-09-13T05:04:16.348129+0000","last_clean":"2024-09-13T05:04:16.348129+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:16.348129+0000","last_undegraded":"2024-09-13T05:04:16.348129+0000","last_fullsized":"2024-09-13T05:04:16.348129+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982881+0000","last_change":"2024-09-13T05:04:16.982881+0000","last_active":"2024-09-13T05:04:16.982881+0000","last_peered":"2024-09-13T05:04:16.982881+0000","last_clean":"2024-09-13T05:04:16.982881+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:16.982881+0000","last_undegraded":"2024-09-13T05:04:16.982881+0000","last_fullsized":"2024-09-13T05:04:16.982881+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:46.387386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3845,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974449+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:15.974449+0000","last_peered":"2024-09-13T05:04:15.974449+0000","last_clean":"2024-09-13T05:04:15.974449+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:15.974449+0000","last_undegraded":"2024-09-13T05:04:15.974449+0000","last_fullsized":"2024-09-13T05:04:15.974449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3280,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971480+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.971480+0000","last_peered":"2024-09-13T05:04:15.971480+0000","last_clean":"2024-09-13T05:04:15.971480+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.971480+0000","last_undegraded":"2024-09-13T05:04:15.971480+0000","last_fullsized":"2024-09-13T05:04:15.971480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.622},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50708,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321132,"statfs":{"total":96636764160,"available":96584839168,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.498},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68768,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303072,"statfs":{"total":96636764160,"available":96566345728,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.496},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.746},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68764,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303076,"statfs":{"total":96636764160,"available":96566349824,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.378},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.29099999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.371},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.30299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.496},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.371},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":1.6539999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":1.6859999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":1.768}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58904,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312936,"statfs":{"total":96636764160,"available":96576446464,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.317},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.30199999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:24.415 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:24.701 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:24.702 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:24.713 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-13T05:04:23.964478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448456,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526264,"statfs":{"total":773094113280,"available":772634894336,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001840"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2585,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974720+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:15.974720+0000","last_peered":"2024-09-13T05:04:15.974720+0000","last_clean":"2024-09-13T05:04:15.974720+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:15.974720+0000","last_undegraded":"2024-09-13T05:04:15.974720+0000","last_fullsized":"2024-09-13T05:04:15.974720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2887,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975359+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:15.975359+0000","last_peered":"2024-09-13T05:04:15.975359+0000","last_clean":"2024-09-13T05:04:15.975359+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:15.975359+0000","last_undegraded":"2024-09-13T05:04:15.975359+0000","last_fullsized":"2024-09-13T05:04:15.975359+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2665,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977022+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:15.977022+0000","last_peered":"2024-09-13T05:04:15.977022+0000","last_clean":"2024-09-13T05:04:15.977022+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:15.977022+0000","last_undegraded":"2024-09-13T05:04:15.977022+0000","last_fullsized":"2024-09-13T05:04:15.977022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5494,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976175+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:15.976175+0000","last_peered":"2024-09-13T05:04:15.976175+0000","last_clean":"2024-09-13T05:04:15.976175+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:15.976175+0000","last_undegraded":"2024-09-13T05:04:15.976175+0000","last_fullsized":"2024-09-13T05:04:15.976175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5546,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974869+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:15.974869+0000","last_peered":"2024-09-13T05:04:15.974869+0000","last_clean":"2024-09-13T05:04:15.974869+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:15.974869+0000","last_undegraded":"2024-09-13T05:04:15.974869+0000","last_fullsized":"2024-09-13T05:04:15.974869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974392+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:15.974392+0000","last_peered":"2024-09-13T05:04:15.974392+0000","last_clean":"2024-09-13T05:04:15.974392+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:15.974392+0000","last_undegraded":"2024-09-13T05:04:15.974392+0000","last_fullsized":"2024-09-13T05:04:15.974392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2143,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971355+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.971355+0000","last_peered":"2024-09-13T05:04:15.971355+0000","last_clean":"2024-09-13T05:04:15.971355+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.971355+0000","last_undegraded":"2024-09-13T05:04:15.971355+0000","last_fullsized":"2024-09-13T05:04:15.971355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2679,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975612+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:15.975612+0000","last_peered":"2024-09-13T05:04:15.975612+0000","last_clean":"2024-09-13T05:04:15.975612+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:15.975612+0000","last_undegraded":"2024-09-13T05:04:15.975612+0000","last_fullsized":"2024-09-13T05:04:15.975612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2794,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974749+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:15.974749+0000","last_peered":"2024-09-13T05:04:15.974749+0000","last_clean":"2024-09-13T05:04:15.974749+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:15.974749+0000","last_undegraded":"2024-09-13T05:04:15.974749+0000","last_fullsized":"2024-09-13T05:04:15.974749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3277,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976313+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:15.976313+0000","last_peered":"2024-09-13T05:04:15.976313+0000","last_clean":"2024-09-13T05:04:15.976313+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:15.976313+0000","last_undegraded":"2024-09-13T05:04:15.976313+0000","last_fullsized":"2024-09-13T05:04:15.976313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.990775+0000","last_change":"2024-09-13T05:04:16.990775+0000","last_active":"2024-09-13T05:04:16.990775+0000","last_peered":"2024-09-13T05:04:16.990775+0000","last_clean":"2024-09-13T05:04:16.990775+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:04:16.990775+0000","last_undegraded":"2024-09-13T05:04:16.990775+0000","last_fullsized":"2024-09-13T05:04:16.990775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:52:14.353780+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2138,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974417+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:15.974417+0000","last_peered":"2024-09-13T05:04:15.974417+0000","last_clean":"2024-09-13T05:04:15.974417+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:15.974417+0000","last_undegraded":"2024-09-13T05:04:15.974417+0000","last_fullsized":"2024-09-13T05:04:15.974417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3184,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977122+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:15.977122+0000","last_peered":"2024-09-13T05:04:15.977122+0000","last_clean":"2024-09-13T05:04:15.977122+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:15.977122+0000","last_undegraded":"2024-09-13T05:04:15.977122+0000","last_fullsized":"2024-09-13T05:04:15.977122+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4991,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971444+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.971444+0000","last_peered":"2024-09-13T05:04:15.971444+0000","last_clean":"2024-09-13T05:04:15.971444+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.971444+0000","last_undegraded":"2024-09-13T05:04:15.971444+0000","last_fullsized":"2024-09-13T05:04:15.971444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974723+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:15.974723+0000","last_peered":"2024-09-13T05:04:15.974723+0000","last_clean":"2024-09-13T05:04:15.974723+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:15.974723+0000","last_undegraded":"2024-09-13T05:04:15.974723+0000","last_fullsized":"2024-09-13T05:04:15.974723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2551,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976149+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:15.976149+0000","last_peered":"2024-09-13T05:04:15.976149+0000","last_clean":"2024-09-13T05:04:15.976149+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:15.976149+0000","last_undegraded":"2024-09-13T05:04:15.976149+0000","last_fullsized":"2024-09-13T05:04:15.976149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975969+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:15.975969+0000","last_peered":"2024-09-13T05:04:15.975969+0000","last_clean":"2024-09-13T05:04:15.975969+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:15.975969+0000","last_undegraded":"2024-09-13T05:04:15.975969+0000","last_fullsized":"2024-09-13T05:04:15.975969+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3464,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.122024+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:16.122024+0000","last_peered":"2024-09-13T05:04:16.122024+0000","last_clean":"2024-09-13T05:04:16.122024+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:16.122024+0000","last_undegraded":"2024-09-13T05:04:16.122024+0000","last_fullsized":"2024-09-13T05:04:16.122024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2589,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974247+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:15.974247+0000","last_peered":"2024-09-13T05:04:15.974247+0000","last_clean":"2024-09-13T05:04:15.974247+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:15.974247+0000","last_undegraded":"2024-09-13T05:04:15.974247+0000","last_fullsized":"2024-09-13T05:04:15.974247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":228,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986088+0000","last_change":"2024-09-13T05:04:16.986088+0000","last_active":"2024-09-13T05:04:16.986088+0000","last_peered":"2024-09-13T05:04:16.986088+0000","last_clean":"2024-09-13T05:04:16.986088+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:16.986088+0000","last_undegraded":"2024-09-13T05:04:16.986088+0000","last_fullsized":"2024-09-13T05:04:16.986088+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":133,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971382+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.971382+0000","last_peered":"2024-09-13T05:04:15.971382+0000","last_clean":"2024-09-13T05:04:15.971382+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.971382+0000","last_undegraded":"2024-09-13T05:04:15.971382+0000","last_fullsized":"2024-09-13T05:04:15.971382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4225,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971385+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.971385+0000","last_peered":"2024-09-13T05:04:15.971385+0000","last_clean":"2024-09-13T05:04:15.971385+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.971385+0000","last_undegraded":"2024-09-13T05:04:15.971385+0000","last_fullsized":"2024-09-13T05:04:15.971385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":85,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.988998+0000","last_change":"2024-09-13T05:04:16.988998+0000","last_active":"2024-09-13T05:04:16.988998+0000","last_peered":"2024-09-13T05:04:16.988998+0000","last_clean":"2024-09-13T05:04:16.988998+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:16.988998+0000","last_undegraded":"2024-09-13T05:04:16.988998+0000","last_fullsized":"2024-09-13T05:04:16.988998+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.991471+0000","last_change":"2024-09-13T05:04:16.991471+0000","last_active":"2024-09-13T05:04:16.991471+0000","last_peered":"2024-09-13T05:04:16.991471+0000","last_clean":"2024-09-13T05:04:16.991471+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:16.991471+0000","last_undegraded":"2024-09-13T05:04:16.991471+0000","last_fullsized":"2024-09-13T05:04:16.991471+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976066+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:15.976066+0000","last_peered":"2024-09-13T05:04:15.976066+0000","last_clean":"2024-09-13T05:04:15.976066+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:15.976066+0000","last_undegraded":"2024-09-13T05:04:15.976066+0000","last_fullsized":"2024-09-13T05:04:15.976066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3913,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975702+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:15.975702+0000","last_peered":"2024-09-13T05:04:15.975702+0000","last_clean":"2024-09-13T05:04:15.975702+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:15.975702+0000","last_undegraded":"2024-09-13T05:04:15.975702+0000","last_fullsized":"2024-09-13T05:04:15.975702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.989101+0000","last_change":"2024-09-13T05:04:16.989101+0000","last_active":"2024-09-13T05:04:16.989101+0000","last_peered":"2024-09-13T05:04:16.989101+0000","last_clean":"2024-09-13T05:04:16.989101+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:16.989101+0000","last_undegraded":"2024-09-13T05:04:16.989101+0000","last_fullsized":"2024-09-13T05:04:16.989101+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3733,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975603+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:15.975603+0000","last_peered":"2024-09-13T05:04:15.975603+0000","last_clean":"2024-09-13T05:04:15.975603+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:15.975603+0000","last_undegraded":"2024-09-13T05:04:15.975603+0000","last_fullsized":"2024-09-13T05:04:15.975603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2875,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118779+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:16.118779+0000","last_peered":"2024-09-13T05:04:16.118779+0000","last_clean":"2024-09-13T05:04:16.118779+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:16.118779+0000","last_undegraded":"2024-09-13T05:04:16.118779+0000","last_fullsized":"2024-09-13T05:04:16.118779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4659,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348167+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.348167+0000","last_peered":"2024-09-13T05:04:16.348167+0000","last_clean":"2024-09-13T05:04:16.348167+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.348167+0000","last_undegraded":"2024-09-13T05:04:16.348167+0000","last_fullsized":"2024-09-13T05:04:16.348167+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2828,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975851+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:15.975851+0000","last_peered":"2024-09-13T05:04:15.975851+0000","last_clean":"2024-09-13T05:04:15.975851+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:15.975851+0000","last_undegraded":"2024-09-13T05:04:15.975851+0000","last_fullsized":"2024-09-13T05:04:15.975851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3802,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982535+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:16.982535+0000","last_peered":"2024-09-13T05:04:16.982535+0000","last_clean":"2024-09-13T05:04:16.982535+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:16.982535+0000","last_undegraded":"2024-09-13T05:04:16.982535+0000","last_fullsized":"2024-09-13T05:04:16.982535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5292,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986129+0000","last_change":"2024-09-13T05:04:16.985647+0000","last_active":"2024-09-13T05:04:16.986129+0000","last_peered":"2024-09-13T05:04:16.986129+0000","last_clean":"2024-09-13T05:04:16.986129+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:04:16.986129+0000","last_undegraded":"2024-09-13T05:04:16.986129+0000","last_fullsized":"2024-09-13T05:04:16.986129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:39.134756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5611,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984699+0000","last_change":"2024-09-13T05:04:16.984083+0000","last_active":"2024-09-13T05:04:16.984699+0000","last_peered":"2024-09-13T05:04:16.984699+0000","last_clean":"2024-09-13T05:04:16.984699+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:16.984699+0000","last_undegraded":"2024-09-13T05:04:16.984699+0000","last_fullsized":"2024-09-13T05:04:16.984699+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:51.096635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3036,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984387+0000","last_change":"2024-09-13T05:04:16.984387+0000","last_active":"2024-09-13T05:04:16.984387+0000","last_peered":"2024-09-13T05:04:16.984387+0000","last_clean":"2024-09-13T05:04:16.984387+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:16.984387+0000","last_undegraded":"2024-09-13T05:04:16.984387+0000","last_fullsized":"2024-09-13T05:04:16.984387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3102,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348129+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:16.348129+0000","last_peered":"2024-09-13T05:04:16.348129+0000","last_clean":"2024-09-13T05:04:16.348129+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:16.348129+0000","last_undegraded":"2024-09-13T05:04:16.348129+0000","last_fullsized":"2024-09-13T05:04:16.348129+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982881+0000","last_change":"2024-09-13T05:04:16.982881+0000","last_active":"2024-09-13T05:04:16.982881+0000","last_peered":"2024-09-13T05:04:16.982881+0000","last_clean":"2024-09-13T05:04:16.982881+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:16.982881+0000","last_undegraded":"2024-09-13T05:04:16.982881+0000","last_fullsized":"2024-09-13T05:04:16.982881+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:46.387386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3845,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974449+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:15.974449+0000","last_peered":"2024-09-13T05:04:15.974449+0000","last_clean":"2024-09-13T05:04:15.974449+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:15.974449+0000","last_undegraded":"2024-09-13T05:04:15.974449+0000","last_fullsized":"2024-09-13T05:04:15.974449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3280,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971480+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.971480+0000","last_peered":"2024-09-13T05:04:15.971480+0000","last_clean":"2024-09-13T05:04:15.971480+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.971480+0000","last_undegraded":"2024-09-13T05:04:15.971480+0000","last_fullsized":"2024-09-13T05:04:15.971480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.622},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50708,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321132,"statfs":{"total":96636764160,"available":96584839168,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.498},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68768,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303072,"statfs":{"total":96636764160,"available":96566345728,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.496},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.746},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68764,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303076,"statfs":{"total":96636764160,"available":96566349824,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.378},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.29099999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.371},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.30299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.496},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.371},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":1.6539999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":1.6859999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":1.768}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58904,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312936,"statfs":{"total":96636764160,"available":96576446464,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.317},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.30199999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:24.717 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-13T05:04:24.717 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:25.003 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:25.004 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:25.015 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-13T05:04:23.964478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448456,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526264,"statfs":{"total":773094113280,"available":772634894336,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001840"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2585,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974720+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:15.974720+0000","last_peered":"2024-09-13T05:04:15.974720+0000","last_clean":"2024-09-13T05:04:15.974720+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:15.974720+0000","last_undegraded":"2024-09-13T05:04:15.974720+0000","last_fullsized":"2024-09-13T05:04:15.974720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2887,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975359+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:15.975359+0000","last_peered":"2024-09-13T05:04:15.975359+0000","last_clean":"2024-09-13T05:04:15.975359+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:15.975359+0000","last_undegraded":"2024-09-13T05:04:15.975359+0000","last_fullsized":"2024-09-13T05:04:15.975359+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2665,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977022+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:15.977022+0000","last_peered":"2024-09-13T05:04:15.977022+0000","last_clean":"2024-09-13T05:04:15.977022+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:15.977022+0000","last_undegraded":"2024-09-13T05:04:15.977022+0000","last_fullsized":"2024-09-13T05:04:15.977022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5494,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976175+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:15.976175+0000","last_peered":"2024-09-13T05:04:15.976175+0000","last_clean":"2024-09-13T05:04:15.976175+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:15.976175+0000","last_undegraded":"2024-09-13T05:04:15.976175+0000","last_fullsized":"2024-09-13T05:04:15.976175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5546,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974869+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:15.974869+0000","last_peered":"2024-09-13T05:04:15.974869+0000","last_clean":"2024-09-13T05:04:15.974869+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:15.974869+0000","last_undegraded":"2024-09-13T05:04:15.974869+0000","last_fullsized":"2024-09-13T05:04:15.974869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974392+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:15.974392+0000","last_peered":"2024-09-13T05:04:15.974392+0000","last_clean":"2024-09-13T05:04:15.974392+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:15.974392+0000","last_undegraded":"2024-09-13T05:04:15.974392+0000","last_fullsized":"2024-09-13T05:04:15.974392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2143,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971355+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.971355+0000","last_peered":"2024-09-13T05:04:15.971355+0000","last_clean":"2024-09-13T05:04:15.971355+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.971355+0000","last_undegraded":"2024-09-13T05:04:15.971355+0000","last_fullsized":"2024-09-13T05:04:15.971355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2679,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975612+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:15.975612+0000","last_peered":"2024-09-13T05:04:15.975612+0000","last_clean":"2024-09-13T05:04:15.975612+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:15.975612+0000","last_undegraded":"2024-09-13T05:04:15.975612+0000","last_fullsized":"2024-09-13T05:04:15.975612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2794,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974749+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:15.974749+0000","last_peered":"2024-09-13T05:04:15.974749+0000","last_clean":"2024-09-13T05:04:15.974749+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:15.974749+0000","last_undegraded":"2024-09-13T05:04:15.974749+0000","last_fullsized":"2024-09-13T05:04:15.974749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3277,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976313+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:15.976313+0000","last_peered":"2024-09-13T05:04:15.976313+0000","last_clean":"2024-09-13T05:04:15.976313+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:15.976313+0000","last_undegraded":"2024-09-13T05:04:15.976313+0000","last_fullsized":"2024-09-13T05:04:15.976313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.990775+0000","last_change":"2024-09-13T05:04:16.990775+0000","last_active":"2024-09-13T05:04:16.990775+0000","last_peered":"2024-09-13T05:04:16.990775+0000","last_clean":"2024-09-13T05:04:16.990775+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:04:16.990775+0000","last_undegraded":"2024-09-13T05:04:16.990775+0000","last_fullsized":"2024-09-13T05:04:16.990775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:52:14.353780+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2138,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974417+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:15.974417+0000","last_peered":"2024-09-13T05:04:15.974417+0000","last_clean":"2024-09-13T05:04:15.974417+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:15.974417+0000","last_undegraded":"2024-09-13T05:04:15.974417+0000","last_fullsized":"2024-09-13T05:04:15.974417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3184,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977122+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:15.977122+0000","last_peered":"2024-09-13T05:04:15.977122+0000","last_clean":"2024-09-13T05:04:15.977122+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:15.977122+0000","last_undegraded":"2024-09-13T05:04:15.977122+0000","last_fullsized":"2024-09-13T05:04:15.977122+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4991,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971444+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.971444+0000","last_peered":"2024-09-13T05:04:15.971444+0000","last_clean":"2024-09-13T05:04:15.971444+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.971444+0000","last_undegraded":"2024-09-13T05:04:15.971444+0000","last_fullsized":"2024-09-13T05:04:15.971444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974723+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:15.974723+0000","last_peered":"2024-09-13T05:04:15.974723+0000","last_clean":"2024-09-13T05:04:15.974723+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:15.974723+0000","last_undegraded":"2024-09-13T05:04:15.974723+0000","last_fullsized":"2024-09-13T05:04:15.974723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2551,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976149+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:15.976149+0000","last_peered":"2024-09-13T05:04:15.976149+0000","last_clean":"2024-09-13T05:04:15.976149+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:15.976149+0000","last_undegraded":"2024-09-13T05:04:15.976149+0000","last_fullsized":"2024-09-13T05:04:15.976149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975969+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:15.975969+0000","last_peered":"2024-09-13T05:04:15.975969+0000","last_clean":"2024-09-13T05:04:15.975969+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:15.975969+0000","last_undegraded":"2024-09-13T05:04:15.975969+0000","last_fullsized":"2024-09-13T05:04:15.975969+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3464,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.122024+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:16.122024+0000","last_peered":"2024-09-13T05:04:16.122024+0000","last_clean":"2024-09-13T05:04:16.122024+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:16.122024+0000","last_undegraded":"2024-09-13T05:04:16.122024+0000","last_fullsized":"2024-09-13T05:04:16.122024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2589,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974247+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:15.974247+0000","last_peered":"2024-09-13T05:04:15.974247+0000","last_clean":"2024-09-13T05:04:15.974247+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:15.974247+0000","last_undegraded":"2024-09-13T05:04:15.974247+0000","last_fullsized":"2024-09-13T05:04:15.974247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":228,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986088+0000","last_change":"2024-09-13T05:04:16.986088+0000","last_active":"2024-09-13T05:04:16.986088+0000","last_peered":"2024-09-13T05:04:16.986088+0000","last_clean":"2024-09-13T05:04:16.986088+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:16.986088+0000","last_undegraded":"2024-09-13T05:04:16.986088+0000","last_fullsized":"2024-09-13T05:04:16.986088+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":133,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971382+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.971382+0000","last_peered":"2024-09-13T05:04:15.971382+0000","last_clean":"2024-09-13T05:04:15.971382+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.971382+0000","last_undegraded":"2024-09-13T05:04:15.971382+0000","last_fullsized":"2024-09-13T05:04:15.971382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4225,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971385+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.971385+0000","last_peered":"2024-09-13T05:04:15.971385+0000","last_clean":"2024-09-13T05:04:15.971385+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.971385+0000","last_undegraded":"2024-09-13T05:04:15.971385+0000","last_fullsized":"2024-09-13T05:04:15.971385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":85,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.988998+0000","last_change":"2024-09-13T05:04:16.988998+0000","last_active":"2024-09-13T05:04:16.988998+0000","last_peered":"2024-09-13T05:04:16.988998+0000","last_clean":"2024-09-13T05:04:16.988998+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:16.988998+0000","last_undegraded":"2024-09-13T05:04:16.988998+0000","last_fullsized":"2024-09-13T05:04:16.988998+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.991471+0000","last_change":"2024-09-13T05:04:16.991471+0000","last_active":"2024-09-13T05:04:16.991471+0000","last_peered":"2024-09-13T05:04:16.991471+0000","last_clean":"2024-09-13T05:04:16.991471+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:16.991471+0000","last_undegraded":"2024-09-13T05:04:16.991471+0000","last_fullsized":"2024-09-13T05:04:16.991471+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976066+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:15.976066+0000","last_peered":"2024-09-13T05:04:15.976066+0000","last_clean":"2024-09-13T05:04:15.976066+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:15.976066+0000","last_undegraded":"2024-09-13T05:04:15.976066+0000","last_fullsized":"2024-09-13T05:04:15.976066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3913,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975702+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:15.975702+0000","last_peered":"2024-09-13T05:04:15.975702+0000","last_clean":"2024-09-13T05:04:15.975702+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:15.975702+0000","last_undegraded":"2024-09-13T05:04:15.975702+0000","last_fullsized":"2024-09-13T05:04:15.975702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.989101+0000","last_change":"2024-09-13T05:04:16.989101+0000","last_active":"2024-09-13T05:04:16.989101+0000","last_peered":"2024-09-13T05:04:16.989101+0000","last_clean":"2024-09-13T05:04:16.989101+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:16.989101+0000","last_undegraded":"2024-09-13T05:04:16.989101+0000","last_fullsized":"2024-09-13T05:04:16.989101+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3733,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975603+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:15.975603+0000","last_peered":"2024-09-13T05:04:15.975603+0000","last_clean":"2024-09-13T05:04:15.975603+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:15.975603+0000","last_undegraded":"2024-09-13T05:04:15.975603+0000","last_fullsized":"2024-09-13T05:04:15.975603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2875,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118779+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:16.118779+0000","last_peered":"2024-09-13T05:04:16.118779+0000","last_clean":"2024-09-13T05:04:16.118779+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:16.118779+0000","last_undegraded":"2024-09-13T05:04:16.118779+0000","last_fullsized":"2024-09-13T05:04:16.118779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4659,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348167+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.348167+0000","last_peered":"2024-09-13T05:04:16.348167+0000","last_clean":"2024-09-13T05:04:16.348167+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.348167+0000","last_undegraded":"2024-09-13T05:04:16.348167+0000","last_fullsized":"2024-09-13T05:04:16.348167+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2828,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975851+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:15.975851+0000","last_peered":"2024-09-13T05:04:15.975851+0000","last_clean":"2024-09-13T05:04:15.975851+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:15.975851+0000","last_undegraded":"2024-09-13T05:04:15.975851+0000","last_fullsized":"2024-09-13T05:04:15.975851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3802,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982535+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:16.982535+0000","last_peered":"2024-09-13T05:04:16.982535+0000","last_clean":"2024-09-13T05:04:16.982535+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:16.982535+0000","last_undegraded":"2024-09-13T05:04:16.982535+0000","last_fullsized":"2024-09-13T05:04:16.982535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5292,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986129+0000","last_change":"2024-09-13T05:04:16.985647+0000","last_active":"2024-09-13T05:04:16.986129+0000","last_peered":"2024-09-13T05:04:16.986129+0000","last_clean":"2024-09-13T05:04:16.986129+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:04:16.986129+0000","last_undegraded":"2024-09-13T05:04:16.986129+0000","last_fullsized":"2024-09-13T05:04:16.986129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:39.134756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5611,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984699+0000","last_change":"2024-09-13T05:04:16.984083+0000","last_active":"2024-09-13T05:04:16.984699+0000","last_peered":"2024-09-13T05:04:16.984699+0000","last_clean":"2024-09-13T05:04:16.984699+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:16.984699+0000","last_undegraded":"2024-09-13T05:04:16.984699+0000","last_fullsized":"2024-09-13T05:04:16.984699+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:51.096635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3036,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984387+0000","last_change":"2024-09-13T05:04:16.984387+0000","last_active":"2024-09-13T05:04:16.984387+0000","last_peered":"2024-09-13T05:04:16.984387+0000","last_clean":"2024-09-13T05:04:16.984387+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:16.984387+0000","last_undegraded":"2024-09-13T05:04:16.984387+0000","last_fullsized":"2024-09-13T05:04:16.984387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3102,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348129+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:16.348129+0000","last_peered":"2024-09-13T05:04:16.348129+0000","last_clean":"2024-09-13T05:04:16.348129+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:16.348129+0000","last_undegraded":"2024-09-13T05:04:16.348129+0000","last_fullsized":"2024-09-13T05:04:16.348129+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4241,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982881+0000","last_change":"2024-09-13T05:04:16.982881+0000","last_active":"2024-09-13T05:04:16.982881+0000","last_peered":"2024-09-13T05:04:16.982881+0000","last_clean":"2024-09-13T05:04:16.982881+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:16.982881+0000","last_undegraded":"2024-09-13T05:04:16.982881+0000","last_fullsized":"2024-09-13T05:04:16.982881+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:46.387386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3845,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974449+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:15.974449+0000","last_peered":"2024-09-13T05:04:15.974449+0000","last_clean":"2024-09-13T05:04:15.974449+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:15.974449+0000","last_undegraded":"2024-09-13T05:04:15.974449+0000","last_fullsized":"2024-09-13T05:04:15.974449+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3280,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971480+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:15.971480+0000","last_peered":"2024-09-13T05:04:15.971480+0000","last_clean":"2024-09-13T05:04:15.971480+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:15.971480+0000","last_undegraded":"2024-09-13T05:04:15.971480+0000","last_fullsized":"2024-09-13T05:04:15.971480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.502},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.375}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.42199999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45300000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56899999999999995}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.51600000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.54700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.60199999999999998}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.622},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.67200000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.68300000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50708,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321132,"statfs":{"total":96636764160,"available":96584839168,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.45400000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.498},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.61699999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.50900000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.68100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68768,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303072,"statfs":{"total":96636764160,"available":96566345728,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.496},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.64800000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.55900000000000005}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.746},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68764,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303076,"statfs":{"total":96636764160,"available":96566349824,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.378},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.29099999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55300000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.58199999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.28199999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.371},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.32400000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.30299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.496},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59799999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.53000000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.371},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":1.6539999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":1.6859999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":1.7110000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":1.768}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.95299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58904,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312936,"statfs":{"total":96636764160,"available":96576446464,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.317},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.30199999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.72899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.66800000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:25.018 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-13T05:04:25.305 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:25.305 INFO:teuthology.orchestra.run.smithi053.stdout:{"epoch":132,"fsid":"01e1476b-1467-4e0e-b01c-0f8cdf8276fe","created":"2024-09-13T05:01:45.075461+0000","modified":"2024-09-13T05:04:16.973441+0000","last_up_change":"2024-09-13T05:01:55.160352+0000","last_in_change":"2024-09-13T05:03:59.880985+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-13T05:01:55.817858+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-13T05:01:55.981425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-13T05:02:05.508713+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":79,"snap_epoch":109,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"baf16a5c-4204-424b-8d8e-a5b1486aa052","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6816","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6817","nonce":1769628429}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6818","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6819","nonce":1769628429}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6822","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6823","nonce":1769628429}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6820","nonce":1769628429},{"type":"v1","addr":"172.21.15.53:6821","nonce":1769628429}]},"public_addr":"172.21.15.53:6817/1769628429","cluster_addr":"172.21.15.53:6819/1769628429","heartbeat_back_addr":"172.21.15.53:6823/1769628429","heartbeat_front_addr":"172.21.15.53:6821/1769628429","state":["exists","up"]},{"osd":1,"uuid":"f3df9a0f-3a9e-4665-9233-718bc0fe1223","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6800","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6801","nonce":827277153}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6802","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6803","nonce":827277153}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6806","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6807","nonce":827277153}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6804","nonce":827277153},{"type":"v1","addr":"172.21.15.53:6805","nonce":827277153}]},"public_addr":"172.21.15.53:6801/827277153","cluster_addr":"172.21.15.53:6803/827277153","heartbeat_back_addr":"172.21.15.53:6807/827277153","heartbeat_front_addr":"172.21.15.53:6805/827277153","state":["exists","up"]},{"osd":2,"uuid":"1304a876-75c5-454b-a60b-8c9823c4aca8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6808","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6809","nonce":1334123504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6810","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6811","nonce":1334123504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6814","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6815","nonce":1334123504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6812","nonce":1334123504},{"type":"v1","addr":"172.21.15.53:6813","nonce":1334123504}]},"public_addr":"172.21.15.53:6809/1334123504","cluster_addr":"172.21.15.53:6811/1334123504","heartbeat_back_addr":"172.21.15.53:6815/1334123504","heartbeat_front_addr":"172.21.15.53:6813/1334123504","state":["exists","up"]},{"osd":3,"uuid":"83668012-8ddb-4fe5-bb31-6e78d2723f12","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6824","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6825","nonce":4134389313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6826","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6827","nonce":4134389313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6830","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6831","nonce":4134389313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.53:6828","nonce":4134389313},{"type":"v1","addr":"172.21.15.53:6829","nonce":4134389313}]},"public_addr":"172.21.15.53:6825/4134389313","cluster_addr":"172.21.15.53:6827/4134389313","heartbeat_back_addr":"172.21.15.53:6831/4134389313","heartbeat_front_addr":"172.21.15.53:6829/4134389313","state":["exists","up"]},{"osd":4,"uuid":"f462232f-5144-4652-9139-9039d53e191a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":117,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6809","nonce":3681797816}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6811","nonce":3681797816}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6815","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6816","nonce":3681797816}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":3681797816},{"type":"v1","addr":"172.21.15.110:6813","nonce":3681797816}]},"public_addr":"172.21.15.110:6809/3681797816","cluster_addr":"172.21.15.110:6811/3681797816","heartbeat_back_addr":"172.21.15.110:6816/3681797816","heartbeat_front_addr":"172.21.15.110:6813/3681797816","state":["exists","up"]},{"osd":5,"uuid":"1d2d8607-f1d9-45e3-8486-3f5bb23b516b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6801","nonce":81482652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6803","nonce":81482652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6807","nonce":81482652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":81482652},{"type":"v1","addr":"172.21.15.110:6805","nonce":81482652}]},"public_addr":"172.21.15.110:6801/81482652","cluster_addr":"172.21.15.110:6803/81482652","heartbeat_back_addr":"172.21.15.110:6807/81482652","heartbeat_front_addr":"172.21.15.110:6805/81482652","state":["exists","up"]},{"osd":6,"uuid":"7fe0fb7d-0a40-4666-82a1-4e18f53b445c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6817","nonce":576600432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6819","nonce":576600432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6823","nonce":576600432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":576600432},{"type":"v1","addr":"172.21.15.110:6821","nonce":576600432}]},"public_addr":"172.21.15.110:6817/576600432","cluster_addr":"172.21.15.110:6819/576600432","heartbeat_back_addr":"172.21.15.110:6823/576600432","heartbeat_front_addr":"172.21.15.110:6821/576600432","state":["exists","up"]},{"osd":7,"uuid":"811503be-abe3-4d64-a094-09ff675f3666","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":131,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6825","nonce":896779674}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6827","nonce":896779674}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6831","nonce":896779674}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":896779674},{"type":"v1","addr":"172.21.15.110:6829","nonce":896779674}]},"public_addr":"172.21.15.110:6825/896779674","cluster_addr":"172.21.15.110:6827/896779674","heartbeat_back_addr":"172.21.15.110:6831/896779674","heartbeat_front_addr":"172.21.15.110:6829/896779674","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358655+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.343203+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.307081+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.315153+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.280902+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.302500+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.354685+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-13T05:01:52.358223+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.3","mappings":[{"from":4,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-13T05:04:26.319 INFO:tasks.ceph:Scrubbing osd.0 2024-09-13T05:04:26.319 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:26.458 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:26.458 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:26.469 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-13T05:04:26.754 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 0 to deep-scrub 2024-09-13T05:04:26.767 INFO:tasks.ceph:Scrubbing osd.1 2024-09-13T05:04:26.767 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:26.892 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:26.893 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:26.893 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:26.903 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-13T05:04:27.188 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 1 to deep-scrub 2024-09-13T05:04:27.201 INFO:tasks.ceph:Scrubbing osd.2 2024-09-13T05:04:27.201 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:27.326 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:27.326 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:27.326 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:27.337 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-13T05:04:27.620 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 2 to deep-scrub 2024-09-13T05:04:27.633 INFO:tasks.ceph:Scrubbing osd.3 2024-09-13T05:04:27.633 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:27.759 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:27.759 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:27.759 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:27.769 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-13T05:04:28.054 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 3 to deep-scrub 2024-09-13T05:04:28.067 INFO:tasks.ceph:Scrubbing osd.4 2024-09-13T05:04:28.067 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:28.197 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:28.197 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:28.197 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:28.207 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-13T05:04:28.491 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 4 to deep-scrub 2024-09-13T05:04:28.503 INFO:tasks.ceph:Scrubbing osd.5 2024-09-13T05:04:28.504 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:28.631 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:28.631 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:28.631 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:28.641 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-13T05:04:28.925 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 5 to deep-scrub 2024-09-13T05:04:28.938 INFO:tasks.ceph:Scrubbing osd.6 2024-09-13T05:04:28.938 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:29.065 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:29.065 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:29.065 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:29.076 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-13T05:04:29.359 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 6 to deep-scrub 2024-09-13T05:04:29.372 INFO:tasks.ceph:Scrubbing osd.7 2024-09-13T05:04:29.372 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-13T05:04:29.498 INFO:teuthology.orchestra.run.smithi053.stdout:{ 2024-09-13T05:04:29.498 INFO:teuthology.orchestra.run.smithi053.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-13T05:04:29.498 INFO:teuthology.orchestra.run.smithi053.stdout:} 2024-09-13T05:04:29.509 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-13T05:04:29.875 INFO:teuthology.orchestra.run.smithi053.stderr:instructed osd(s) 7 to deep-scrub 2024-09-13T05:04:29.888 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:30.178 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:30.179 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:30.189 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-13T05:04:29.965813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20266,"ondisk_log_size":20266,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448456,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":435827,"kb_avail":754526264,"statfs":{"total":773094113280,"available":772634894336,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":446287417},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002895"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2585,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974720+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:15.974720+0000","last_peered":"2024-09-13T05:04:15.974720+0000","last_clean":"2024-09-13T05:04:15.974720+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:15.974720+0000","last_undegraded":"2024-09-13T05:04:15.974720+0000","last_fullsized":"2024-09-13T05:04:15.974720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:45:41.684623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"101'555","reported_seq":2887,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975359+0000","last_change":"2024-09-13T05:04:03.920948+0000","last_active":"2024-09-13T05:04:15.975359+0000","last_peered":"2024-09-13T05:04:15.975359+0000","last_clean":"2024-09-13T05:04:15.975359+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:15.975359+0000","last_undegraded":"2024-09-13T05:04:15.975359+0000","last_fullsized":"2024-09-13T05:04:15.975359+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:33:38.721432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2665,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977022+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:15.977022+0000","last_peered":"2024-09-13T05:04:15.977022+0000","last_clean":"2024-09-13T05:04:15.977022+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:15.977022+0000","last_undegraded":"2024-09-13T05:04:15.977022+0000","last_fullsized":"2024-09-13T05:04:15.977022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:02:45.118883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"109'599","reported_seq":5494,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976175+0000","last_change":"2024-09-13T05:03:52.988773+0000","last_active":"2024-09-13T05:04:15.976175+0000","last_peered":"2024-09-13T05:04:15.976175+0000","last_clean":"2024-09-13T05:04:15.976175+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:15.976175+0000","last_undegraded":"2024-09-13T05:04:15.976175+0000","last_fullsized":"2024-09-13T05:04:15.976175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:10:43.852500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5546,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974869+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:15.974869+0000","last_peered":"2024-09-13T05:04:15.974869+0000","last_clean":"2024-09-13T05:04:15.974869+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:15.974869+0000","last_undegraded":"2024-09-13T05:04:15.974869+0000","last_fullsized":"2024-09-13T05:04:15.974869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:07:33.734462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"109'1801","reported_seq":6376,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974392+0000","last_change":"2024-09-13T05:03:52.985526+0000","last_active":"2024-09-13T05:04:15.974392+0000","last_peered":"2024-09-13T05:04:15.974392+0000","last_clean":"2024-09-13T05:04:15.974392+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:15.974392+0000","last_undegraded":"2024-09-13T05:04:15.974392+0000","last_fullsized":"2024-09-13T05:04:15.974392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1801,"log_dups_size":0,"ondisk_log_size":1801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:47.275957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"101'463","reported_seq":2143,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971355+0000","last_change":"2024-09-13T05:04:03.919115+0000","last_active":"2024-09-13T05:04:15.971355+0000","last_peered":"2024-09-13T05:04:15.971355+0000","last_clean":"2024-09-13T05:04:15.971355+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:15.971355+0000","last_undegraded":"2024-09-13T05:04:15.971355+0000","last_fullsized":"2024-09-13T05:04:15.971355+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:43:53.174659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"108'399","reported_seq":2679,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975612+0000","last_change":"2024-09-13T05:04:03.920426+0000","last_active":"2024-09-13T05:04:15.975612+0000","last_peered":"2024-09-13T05:04:15.975612+0000","last_clean":"2024-09-13T05:04:15.975612+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:15.975612+0000","last_undegraded":"2024-09-13T05:04:15.975612+0000","last_fullsized":"2024-09-13T05:04:15.975612+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:52:11.502884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2794,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974749+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:15.974749+0000","last_peered":"2024-09-13T05:04:15.974749+0000","last_clean":"2024-09-13T05:04:15.974749+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:15.974749+0000","last_undegraded":"2024-09-13T05:04:15.974749+0000","last_fullsized":"2024-09-13T05:04:15.974749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:58:53.746189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"109'360","reported_seq":3277,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976313+0000","last_change":"2024-09-13T05:03:52.988253+0000","last_active":"2024-09-13T05:04:15.976313+0000","last_peered":"2024-09-13T05:04:15.976313+0000","last_clean":"2024-09-13T05:04:15.976313+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:15.976313+0000","last_undegraded":"2024-09-13T05:04:15.976313+0000","last_fullsized":"2024-09-13T05:04:15.976313+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:35:16.527694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.990775+0000","last_change":"2024-09-13T05:04:16.990775+0000","last_active":"2024-09-13T05:04:16.990775+0000","last_peered":"2024-09-13T05:04:16.990775+0000","last_clean":"2024-09-13T05:04:16.990775+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:04:16.990775+0000","last_undegraded":"2024-09-13T05:04:16.990775+0000","last_fullsized":"2024-09-13T05:04:16.990775+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:52:14.353780+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"102'348","reported_seq":2139,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.977891+0000","last_change":"2024-09-13T05:03:52.985918+0000","last_active":"2024-09-13T05:04:16.977891+0000","last_peered":"2024-09-13T05:04:16.977891+0000","last_clean":"2024-09-13T05:04:16.977891+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:16.977891+0000","last_undegraded":"2024-09-13T05:04:16.977891+0000","last_fullsized":"2024-09-13T05:04:16.977891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:05:33.165965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970114+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:59:07.618542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"109'468","reported_seq":3184,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.977122+0000","last_change":"2024-09-13T05:04:04.922841+0000","last_active":"2024-09-13T05:04:15.977122+0000","last_peered":"2024-09-13T05:04:15.977122+0000","last_clean":"2024-09-13T05:04:15.977122+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:15.977122+0000","last_undegraded":"2024-09-13T05:04:15.977122+0000","last_fullsized":"2024-09-13T05:04:15.977122+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:58:17.036357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348159+0000","last_change":"2024-09-13T05:04:14.970073+0000","last_active":"2024-09-13T05:04:16.348159+0000","last_peered":"2024-09-13T05:04:16.348159+0000","last_clean":"2024-09-13T05:04:16.348159+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:16.348159+0000","last_undegraded":"2024-09-13T05:04:16.348159+0000","last_fullsized":"2024-09-13T05:04:16.348159+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:20:49.568782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"108'1123","reported_seq":4991,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971444+0000","last_change":"2024-09-13T05:03:52.977104+0000","last_active":"2024-09-13T05:04:15.971444+0000","last_peered":"2024-09-13T05:04:15.971444+0000","last_clean":"2024-09-13T05:04:15.971444+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:15.971444+0000","last_undegraded":"2024-09-13T05:04:15.971444+0000","last_fullsized":"2024-09-13T05:04:15.971444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1123,"log_dups_size":0,"ondisk_log_size":1123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:32:50.044119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":132,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974723+0000","last_change":"2024-09-13T05:01:59.565274+0000","last_active":"2024-09-13T05:04:15.974723+0000","last_peered":"2024-09-13T05:04:15.974723+0000","last_clean":"2024-09-13T05:04:15.974723+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:15.974723+0000","last_undegraded":"2024-09-13T05:04:15.974723+0000","last_fullsized":"2024-09-13T05:04:15.974723+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:34:18.099939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"108'776","reported_seq":2551,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976149+0000","last_change":"2024-09-13T05:04:04.926037+0000","last_active":"2024-09-13T05:04:15.976149+0000","last_peered":"2024-09-13T05:04:15.976149+0000","last_clean":"2024-09-13T05:04:15.976149+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:15.976149+0000","last_undegraded":"2024-09-13T05:04:15.976149+0000","last_fullsized":"2024-09-13T05:04:15.976149+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":776,"log_dups_size":0,"ondisk_log_size":776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:12:22.381267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975969+0000","last_change":"2024-09-13T05:04:00.893816+0000","last_active":"2024-09-13T05:04:15.975969+0000","last_peered":"2024-09-13T05:04:15.975969+0000","last_clean":"2024-09-13T05:04:15.975969+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:15.975969+0000","last_undegraded":"2024-09-13T05:04:15.975969+0000","last_fullsized":"2024-09-13T05:04:15.975969+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:45:39.877052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3464,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.122024+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:16.122024+0000","last_peered":"2024-09-13T05:04:16.122024+0000","last_clean":"2024-09-13T05:04:16.122024+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:16.122024+0000","last_undegraded":"2024-09-13T05:04:16.122024+0000","last_fullsized":"2024-09-13T05:04:16.122024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:19:10.406415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"109'459","reported_seq":2589,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.974247+0000","last_change":"2024-09-13T05:03:52.983930+0000","last_active":"2024-09-13T05:04:15.974247+0000","last_peered":"2024-09-13T05:04:15.974247+0000","last_clean":"2024-09-13T05:04:15.974247+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:15.974247+0000","last_undegraded":"2024-09-13T05:04:15.974247+0000","last_fullsized":"2024-09-13T05:04:15.974247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:57:20.480511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":228,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986088+0000","last_change":"2024-09-13T05:04:16.986088+0000","last_active":"2024-09-13T05:04:16.986088+0000","last_peered":"2024-09-13T05:04:16.986088+0000","last_clean":"2024-09-13T05:04:16.986088+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:16.986088+0000","last_undegraded":"2024-09-13T05:04:16.986088+0000","last_fullsized":"2024-09-13T05:04:16.986088+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:09:00.540010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":133,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971382+0000","last_change":"2024-09-13T05:01:59.195889+0000","last_active":"2024-09-13T05:04:15.971382+0000","last_peered":"2024-09-13T05:04:15.971382+0000","last_clean":"2024-09-13T05:04:15.971382+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:15.971382+0000","last_undegraded":"2024-09-13T05:04:15.971382+0000","last_fullsized":"2024-09-13T05:04:15.971382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:59:39.261335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"106'608","reported_seq":4225,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.971385+0000","last_change":"2024-09-13T05:03:52.976766+0000","last_active":"2024-09-13T05:04:15.971385+0000","last_peered":"2024-09-13T05:04:15.971385+0000","last_clean":"2024-09-13T05:04:15.971385+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:15.971385+0000","last_undegraded":"2024-09-13T05:04:15.971385+0000","last_fullsized":"2024-09-13T05:04:15.971385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:10:24.414503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":85,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.988998+0000","last_change":"2024-09-13T05:04:16.988998+0000","last_active":"2024-09-13T05:04:16.988998+0000","last_peered":"2024-09-13T05:04:16.988998+0000","last_clean":"2024-09-13T05:04:16.988998+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:16.988998+0000","last_undegraded":"2024-09-13T05:04:16.988998+0000","last_fullsized":"2024-09-13T05:04:16.988998+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:12:21.529674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.991471+0000","last_change":"2024-09-13T05:04:16.991471+0000","last_active":"2024-09-13T05:04:16.991471+0000","last_peered":"2024-09-13T05:04:16.991471+0000","last_clean":"2024-09-13T05:04:16.991471+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:16.991471+0000","last_undegraded":"2024-09-13T05:04:16.991471+0000","last_fullsized":"2024-09-13T05:04:16.991471+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:32:25.382297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":131,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.976066+0000","last_change":"2024-09-13T05:01:59.565310+0000","last_active":"2024-09-13T05:04:15.976066+0000","last_peered":"2024-09-13T05:04:15.976066+0000","last_clean":"2024-09-13T05:04:15.976066+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:15.976066+0000","last_undegraded":"2024-09-13T05:04:15.976066+0000","last_fullsized":"2024-09-13T05:04:15.976066+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:48:43.634318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"108'1180","reported_seq":3913,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975702+0000","last_change":"2024-09-13T05:04:04.925066+0000","last_active":"2024-09-13T05:04:15.975702+0000","last_peered":"2024-09-13T05:04:15.975702+0000","last_clean":"2024-09-13T05:04:15.975702+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:15.975702+0000","last_undegraded":"2024-09-13T05:04:15.975702+0000","last_fullsized":"2024-09-13T05:04:15.975702+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":1180,"log_dups_size":0,"ondisk_log_size":1180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:57:33.129148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.989101+0000","last_change":"2024-09-13T05:04:16.989101+0000","last_active":"2024-09-13T05:04:16.989101+0000","last_peered":"2024-09-13T05:04:16.989101+0000","last_clean":"2024-09-13T05:04:16.989101+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:16.989101+0000","last_undegraded":"2024-09-13T05:04:16.989101+0000","last_fullsized":"2024-09-13T05:04:16.989101+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:49:11.943209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"109'446","reported_seq":3733,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975603+0000","last_change":"2024-09-13T05:04:04.925573+0000","last_active":"2024-09-13T05:04:15.975603+0000","last_peered":"2024-09-13T05:04:15.975603+0000","last_clean":"2024-09-13T05:04:15.975603+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:15.975603+0000","last_undegraded":"2024-09-13T05:04:15.975603+0000","last_fullsized":"2024-09-13T05:04:15.975603+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":446,"log_dups_size":0,"ondisk_log_size":446,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:47:32.128666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"109'463","reported_seq":2875,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.118779+0000","last_change":"2024-09-13T05:03:52.985920+0000","last_active":"2024-09-13T05:04:16.118779+0000","last_peered":"2024-09-13T05:04:16.118779+0000","last_clean":"2024-09-13T05:04:16.118779+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:16.118779+0000","last_undegraded":"2024-09-13T05:04:16.118779+0000","last_fullsized":"2024-09-13T05:04:16.118779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:08:38.390524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"108'643","reported_seq":4659,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.348167+0000","last_change":"2024-09-13T05:04:14.972995+0000","last_active":"2024-09-13T05:04:16.348167+0000","last_peered":"2024-09-13T05:04:16.348167+0000","last_clean":"2024-09-13T05:04:16.348167+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:16.348167+0000","last_undegraded":"2024-09-13T05:04:16.348167+0000","last_fullsized":"2024-09-13T05:04:16.348167+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:34:40.899016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"103'885","reported_seq":2828,"reported_epoch":131,"state":"active+clean","last_fresh":"2024-09-13T05:04:15.975851+0000","last_change":"2024-09-13T05:04:04.923667+0000","last_active":"2024-09-13T05:04:15.975851+0000","last_peered":"2024-09-13T05:04:15.975851+0000","last_clean":"2024-09-13T05:04:15.975851+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:15.975851+0000","last_undegraded":"2024-09-13T05:04:15.975851+0000","last_fullsized":"2024-09-13T05:04:15.975851+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:47:57.213696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"103'437","reported_seq":3802,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982535+0000","last_change":"2024-09-13T05:04:03.920358+0000","last_active":"2024-09-13T05:04:16.982535+0000","last_peered":"2024-09-13T05:04:16.982535+0000","last_clean":"2024-09-13T05:04:16.982535+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:16.982535+0000","last_undegraded":"2024-09-13T05:04:16.982535+0000","last_fullsized":"2024-09-13T05:04:16.982535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":437,"log_dups_size":0,"ondisk_log_size":437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:04:45.697564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5292,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.986129+0000","last_change":"2024-09-13T05:04:16.985647+0000","last_active":"2024-09-13T05:04:16.986129+0000","last_peered":"2024-09-13T05:04:16.986129+0000","last_clean":"2024-09-13T05:04:16.986129+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:04:16.986129+0000","last_undegraded":"2024-09-13T05:04:16.986129+0000","last_fullsized":"2024-09-13T05:04:16.986129+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:29:39.134756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"109'575","reported_seq":5611,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984699+0000","last_change":"2024-09-13T05:04:16.984083+0000","last_active":"2024-09-13T05:04:16.984699+0000","last_peered":"2024-09-13T05:04:16.984699+0000","last_clean":"2024-09-13T05:04:16.984699+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:16.984699+0000","last_undegraded":"2024-09-13T05:04:16.984699+0000","last_fullsized":"2024-09-13T05:04:16.984699+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:37:51.096635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"108'390","reported_seq":3036,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984387+0000","last_change":"2024-09-13T05:04:16.984387+0000","last_active":"2024-09-13T05:04:16.984387+0000","last_peered":"2024-09-13T05:04:16.984387+0000","last_clean":"2024-09-13T05:04:16.984387+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:16.984387+0000","last_undegraded":"2024-09-13T05:04:16.984387+0000","last_fullsized":"2024-09-13T05:04:16.984387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:40:22.537303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"108'874","reported_seq":3103,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:17.150932+0000","last_change":"2024-09-13T05:04:14.972140+0000","last_active":"2024-09-13T05:04:17.150932+0000","last_peered":"2024-09-13T05:04:17.150932+0000","last_clean":"2024-09-13T05:04:17.150932+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:17.150932+0000","last_undegraded":"2024-09-13T05:04:17.150932+0000","last_fullsized":"2024-09-13T05:04:17.150932+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:58:47.530985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"109'551","reported_seq":4242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.983501+0000","last_change":"2024-09-13T05:04:16.982881+0000","last_active":"2024-09-13T05:04:16.983501+0000","last_peered":"2024-09-13T05:04:16.983501+0000","last_clean":"2024-09-13T05:04:16.983501+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:16.983501+0000","last_undegraded":"2024-09-13T05:04:16.983501+0000","last_fullsized":"2024-09-13T05:04:16.983501+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:50:46.387386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3846,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.983799+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:16.983799+0000","last_peered":"2024-09-13T05:04:16.983799+0000","last_clean":"2024-09-13T05:04:16.983799+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:16.983799+0000","last_undegraded":"2024-09-13T05:04:16.983799+0000","last_fullsized":"2024-09-13T05:04:16.983799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:21:23.792865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"109'761","reported_seq":3281,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.979927+0000","last_change":"2024-09-13T05:03:53.030691+0000","last_active":"2024-09-13T05:04:16.979927+0000","last_peered":"2024-09-13T05:04:16.979927+0000","last_clean":"2024-09-13T05:04:16.979927+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:16.979927+0000","last_undegraded":"2024-09-13T05:04:16.979927+0000","last_fullsized":"2024-09-13T05:04:16.979927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:32:59.795908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20146,"ondisk_log_size":20146,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58968,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312872,"statfs":{"total":96636764160,"available":96576380928,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.41699999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.505},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.53000000000000003}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.63}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40852,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330988,"statfs":{"total":96636764160,"available":96594931712,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.44500000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.49099999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.53400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50708,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94321132,"statfs":{"total":96636764160,"available":96584839168,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50330054},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.73899999999999999}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.59599999999999997}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.378},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.41199999999999998}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.53800000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68768,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303072,"statfs":{"total":96636764160,"available":96566345728,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.372},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.48899999999999999}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.432},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.45100000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.41899999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.47899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.59199999999999997}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.53300000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68764,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303076,"statfs":{"total":96636764160,"available":96566349824,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.35799999999999998}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.38},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.442}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.55900000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.58199999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.51100000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.69099999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28572,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343268,"statfs":{"total":96636764160,"available":96607506432,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.312}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.433}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.44500000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.51200000000000001}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.622}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.55700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72920,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298920,"statfs":{"total":96636764160,"available":96562094080,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.30599999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.38}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.54500000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.76500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58904,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312936,"statfs":{"total":96636764160,"available":96576446464,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.621}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.76100000000000001}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.73899999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.995}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:30.195 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.196 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.197 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.198 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.199 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:30.200 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T05:04:50.202 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:04:50.500 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:04:50.500 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:04:50.512 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-13T05:04:49.970616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20515,"ondisk_log_size":20515,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":452480,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":439923,"kb_avail":754522240,"statfs":{"total":773094113280,"available":772630773760,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":450481721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003006"},"pg_stats":[{"pgid":"3.1f","version":"109'358","reported_seq":2586,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.984431+0000","last_change":"2024-09-13T05:03:52.984313+0000","last_active":"2024-09-13T05:04:16.984431+0000","last_peered":"2024-09-13T05:04:16.984431+0000","last_clean":"2024-09-13T05:04:16.984431+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:04:16.984431+0000","last_undegraded":"2024-09-13T05:04:16.984431+0000","last_fullsized":"2024-09-13T05:04:16.984431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":358,"log_dups_size":0,"ondisk_log_size":358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"132'562","reported_seq":2906,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:28.957842+0000","last_change":"2024-09-13T05:04:28.957923+0000","last_active":"2024-09-13T05:04:28.957842+0000","last_peered":"2024-09-13T05:04:28.957842+0000","last_clean":"2024-09-13T05:04:28.957842+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:28.957842+0000","last_undegraded":"2024-09-13T05:04:28.957842+0000","last_fullsized":"2024-09-13T05:04:28.957842+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'562","last_scrub_stamp":"2024-09-13T05:04:28.957821+0000","last_deep_scrub":"132'562","last_deep_scrub_stamp":"2024-09-13T05:04:28.957821+0000","last_clean_scrub_stamp":"2024-09-13T05:04:28.957821+0000","objects_scrubbed":7,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:57.988677+0000","scrub_duration":0.0067411629999999997,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"101'511","reported_seq":2667,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:28.925869+0000","last_change":"2024-09-13T05:03:52.988547+0000","last_active":"2024-09-13T05:04:28.925869+0000","last_peered":"2024-09-13T05:04:28.925869+0000","last_clean":"2024-09-13T05:04:28.925869+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:28.925869+0000","last_undegraded":"2024-09-13T05:04:28.925869+0000","last_fullsized":"2024-09-13T05:04:28.925869+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"132'609","reported_seq":5517,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:34.250565+0000","last_change":"2024-09-13T05:04:34.250565+0000","last_active":"2024-09-13T05:04:34.250565+0000","last_peered":"2024-09-13T05:04:34.250565+0000","last_clean":"2024-09-13T05:04:34.250565+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:34.250565+0000","last_undegraded":"2024-09-13T05:04:34.250565+0000","last_fullsized":"2024-09-13T05:04:34.250565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'609","last_scrub_stamp":"2024-09-13T05:04:34.250433+0000","last_deep_scrub":"132'609","last_deep_scrub_stamp":"2024-09-13T05:04:34.250433+0000","last_clean_scrub_stamp":"2024-09-13T05:04:34.250433+0000","objects_scrubbed":10,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.749268+0000","scrub_duration":0.0089988020000000002,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"106'729","reported_seq":5548,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:26.754955+0000","last_change":"2024-09-13T05:03:52.984908+0000","last_active":"2024-09-13T05:04:26.754955+0000","last_peered":"2024-09-13T05:04:26.754955+0000","last_clean":"2024-09-13T05:04:26.754955+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:26.754955+0000","last_undegraded":"2024-09-13T05:04:26.754955+0000","last_fullsized":"2024-09-13T05:04:26.754955+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"132'1816","reported_seq":6404,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:41.417700+0000","last_change":"2024-09-13T05:04:41.417831+0000","last_active":"2024-09-13T05:04:41.417700+0000","last_peered":"2024-09-13T05:04:41.417700+0000","last_clean":"2024-09-13T05:04:41.417700+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:41.417700+0000","last_undegraded":"2024-09-13T05:04:41.417700+0000","last_fullsized":"2024-09-13T05:04:41.417700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1816","last_scrub_stamp":"2024-09-13T05:04:41.417678+0000","last_deep_scrub":"132'1816","last_deep_scrub_stamp":"2024-09-13T05:04:41.417678+0000","last_clean_scrub_stamp":"2024-09-13T05:04:41.417678+0000","objects_scrubbed":15,"log_size":1816,"log_dups_size":0,"ondisk_log_size":1816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.745357+0000","scrub_duration":0.0149617,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"132'471","reported_seq":2164,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:33.480073+0000","last_change":"2024-09-13T05:04:33.480186+0000","last_active":"2024-09-13T05:04:33.480073+0000","last_peered":"2024-09-13T05:04:33.480073+0000","last_clean":"2024-09-13T05:04:33.480073+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:33.480073+0000","last_undegraded":"2024-09-13T05:04:33.480073+0000","last_fullsized":"2024-09-13T05:04:33.480073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'471","last_scrub_stamp":"2024-09-13T05:04:33.480048+0000","last_deep_scrub":"132'471","last_deep_scrub_stamp":"2024-09-13T05:04:33.480048+0000","last_clean_scrub_stamp":"2024-09-13T05:04:33.480048+0000","objects_scrubbed":8,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:02.181935+0000","scrub_duration":0.0073536670000000004,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"132'408","reported_seq":2700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:33.055703+0000","last_change":"2024-09-13T05:04:33.055855+0000","last_active":"2024-09-13T05:04:33.055703+0000","last_peered":"2024-09-13T05:04:33.055703+0000","last_clean":"2024-09-13T05:04:33.055703+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:33.055703+0000","last_undegraded":"2024-09-13T05:04:33.055703+0000","last_fullsized":"2024-09-13T05:04:33.055703+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'408","last_scrub_stamp":"2024-09-13T05:04:33.055680+0000","last_deep_scrub":"132'408","last_deep_scrub_stamp":"2024-09-13T05:04:33.055680+0000","last_clean_scrub_stamp":"2024-09-13T05:04:33.055680+0000","objects_scrubbed":9,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:06:43.875417+0000","scrub_duration":0.0083631799999999996,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"109'389","reported_seq":2796,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:26.754946+0000","last_change":"2024-09-13T05:03:52.984415+0000","last_active":"2024-09-13T05:04:26.754946+0000","last_peered":"2024-09-13T05:04:26.754946+0000","last_clean":"2024-09-13T05:04:26.754946+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:26.754946+0000","last_undegraded":"2024-09-13T05:04:26.754946+0000","last_fullsized":"2024-09-13T05:04:26.754946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":389,"log_dups_size":0,"ondisk_log_size":389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"132'367","reported_seq":3297,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:36.280067+0000","last_change":"2024-09-13T05:04:36.280174+0000","last_active":"2024-09-13T05:04:36.280067+0000","last_peered":"2024-09-13T05:04:36.280067+0000","last_clean":"2024-09-13T05:04:36.280067+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:36.280067+0000","last_undegraded":"2024-09-13T05:04:36.280067+0000","last_fullsized":"2024-09-13T05:04:36.280067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'367","last_scrub_stamp":"2024-09-13T05:04:36.280044+0000","last_deep_scrub":"132'367","last_deep_scrub_stamp":"2024-09-13T05:04:36.280044+0000","last_clean_scrub_stamp":"2024-09-13T05:04:36.280044+0000","objects_scrubbed":7,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:01:29.404235+0000","scrub_duration":0.0063987940000000002,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"103'636","reported_seq":3639,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:29.875453+0000","last_change":"2024-09-13T05:04:16.990775+0000","last_active":"2024-09-13T05:04:29.875453+0000","last_peered":"2024-09-13T05:04:29.875453+0000","last_clean":"2024-09-13T05:04:29.875453+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:04:29.875453+0000","last_undegraded":"2024-09-13T05:04:29.875453+0000","last_fullsized":"2024-09-13T05:04:29.875453+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"132'355","reported_seq":2158,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.359014+0000","last_change":"2024-09-13T05:04:38.359143+0000","last_active":"2024-09-13T05:04:38.359014+0000","last_peered":"2024-09-13T05:04:38.359014+0000","last_clean":"2024-09-13T05:04:38.359014+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:38.359014+0000","last_undegraded":"2024-09-13T05:04:38.359014+0000","last_fullsized":"2024-09-13T05:04:38.359014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'355","last_scrub_stamp":"2024-09-13T05:04:38.358982+0000","last_deep_scrub":"132'355","last_deep_scrub_stamp":"2024-09-13T05:04:38.358982+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.358982+0000","objects_scrubbed":7,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:55:12.706645+0000","scrub_duration":0.0064214700000000003,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.094922+0000","last_change":"2024-09-13T05:04:38.095070+0000","last_active":"2024-09-13T05:04:38.094922+0000","last_peered":"2024-09-13T05:04:38.094922+0000","last_clean":"2024-09-13T05:04:38.094922+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:38.094922+0000","last_undegraded":"2024-09-13T05:04:38.094922+0000","last_fullsized":"2024-09-13T05:04:38.094922+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:38.094902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:38.094902+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.094902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:53.100613+0000","scrub_duration":0.001682458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"132'479","reported_seq":3207,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.251889+0000","last_change":"2024-09-13T05:04:38.252027+0000","last_active":"2024-09-13T05:04:38.251889+0000","last_peered":"2024-09-13T05:04:38.251889+0000","last_clean":"2024-09-13T05:04:38.251889+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:38.251889+0000","last_undegraded":"2024-09-13T05:04:38.251889+0000","last_fullsized":"2024-09-13T05:04:38.251889+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'479","last_scrub_stamp":"2024-09-13T05:04:38.251866+0000","last_deep_scrub":"132'479","last_deep_scrub_stamp":"2024-09-13T05:04:38.251866+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.251866+0000","objects_scrubbed":11,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:57:02.597436+0000","scrub_duration":0.0094311050000000004,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:40.074762+0000","last_change":"2024-09-13T05:04:40.074869+0000","last_active":"2024-09-13T05:04:40.074762+0000","last_peered":"2024-09-13T05:04:40.074762+0000","last_clean":"2024-09-13T05:04:40.074762+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:40.074762+0000","last_undegraded":"2024-09-13T05:04:40.074762+0000","last_fullsized":"2024-09-13T05:04:40.074762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:40.074740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:40.074740+0000","last_clean_scrub_stamp":"2024-09-13T05:04:40.074740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:49.156896+0000","scrub_duration":0.0019662630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"132'1133","reported_seq":5013,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:32.522173+0000","last_change":"2024-09-13T05:04:32.522334+0000","last_active":"2024-09-13T05:04:32.522173+0000","last_peered":"2024-09-13T05:04:32.522173+0000","last_clean":"2024-09-13T05:04:32.522173+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:32.522173+0000","last_undegraded":"2024-09-13T05:04:32.522173+0000","last_fullsized":"2024-09-13T05:04:32.522173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1133","last_scrub_stamp":"2024-09-13T05:04:32.522147+0000","last_deep_scrub":"132'1133","last_deep_scrub_stamp":"2024-09-13T05:04:32.522147+0000","last_clean_scrub_stamp":"2024-09-13T05:04:32.522147+0000","objects_scrubbed":10,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:30.658913+0000","scrub_duration":0.010768115,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":144,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:33.399149+0000","last_change":"2024-09-13T05:04:33.399292+0000","last_active":"2024-09-13T05:04:33.399149+0000","last_peered":"2024-09-13T05:04:33.399149+0000","last_clean":"2024-09-13T05:04:33.399149+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:33.399149+0000","last_undegraded":"2024-09-13T05:04:33.399149+0000","last_fullsized":"2024-09-13T05:04:33.399149+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:33.399128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:33.399128+0000","last_clean_scrub_stamp":"2024-09-13T05:04:33.399128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:47.696263+0000","scrub_duration":0.0021928830000000001,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'790","reported_seq":2577,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.010912+0000","last_change":"2024-09-13T05:04:38.011021+0000","last_active":"2024-09-13T05:04:38.010912+0000","last_peered":"2024-09-13T05:04:38.010912+0000","last_clean":"2024-09-13T05:04:38.010912+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:38.010912+0000","last_undegraded":"2024-09-13T05:04:38.010912+0000","last_fullsized":"2024-09-13T05:04:38.010912+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'790","last_scrub_stamp":"2024-09-13T05:04:38.010887+0000","last_deep_scrub":"132'790","last_deep_scrub_stamp":"2024-09-13T05:04:38.010887+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.010887+0000","objects_scrubbed":14,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:41.131532+0000","scrub_duration":0.010720327,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:35.024024+0000","last_change":"2024-09-13T05:04:35.024136+0000","last_active":"2024-09-13T05:04:35.024024+0000","last_peered":"2024-09-13T05:04:35.024024+0000","last_clean":"2024-09-13T05:04:35.024024+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:35.024024+0000","last_undegraded":"2024-09-13T05:04:35.024024+0000","last_fullsized":"2024-09-13T05:04:35.024024+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:35.024003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:35.024003+0000","last_clean_scrub_stamp":"2024-09-13T05:04:35.024003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:50.798100+0000","scrub_duration":0.0017496670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"109'713","reported_seq":3465,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.982862+0000","last_change":"2024-09-13T05:03:52.983967+0000","last_active":"2024-09-13T05:04:16.982862+0000","last_peered":"2024-09-13T05:04:16.982862+0000","last_clean":"2024-09-13T05:04:16.982862+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:16.982862+0000","last_undegraded":"2024-09-13T05:04:16.982862+0000","last_fullsized":"2024-09-13T05:04:16.982862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"132'468","reported_seq":2610,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:28.128304+0000","last_change":"2024-09-13T05:04:28.128424+0000","last_active":"2024-09-13T05:04:28.128304+0000","last_peered":"2024-09-13T05:04:28.128304+0000","last_clean":"2024-09-13T05:04:28.128304+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:28.128304+0000","last_undegraded":"2024-09-13T05:04:28.128304+0000","last_fullsized":"2024-09-13T05:04:28.128304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'468","last_scrub_stamp":"2024-09-13T05:04:28.128285+0000","last_deep_scrub":"132'468","last_deep_scrub_stamp":"2024-09-13T05:04:28.128285+0000","last_clean_scrub_stamp":"2024-09-13T05:04:28.128285+0000","objects_scrubbed":9,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:34:23.536411+0000","scrub_duration":0.0092365989999999999,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"132'119","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:39.078342+0000","last_change":"2024-09-13T05:04:39.078450+0000","last_active":"2024-09-13T05:04:39.078342+0000","last_peered":"2024-09-13T05:04:39.078342+0000","last_clean":"2024-09-13T05:04:39.078342+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:39.078342+0000","last_undegraded":"2024-09-13T05:04:39.078342+0000","last_fullsized":"2024-09-13T05:04:39.078342+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'119","last_scrub_stamp":"2024-09-13T05:04:39.078318+0000","last_deep_scrub":"132'119","last_deep_scrub_stamp":"2024-09-13T05:04:39.078318+0000","last_clean_scrub_stamp":"2024-09-13T05:04:39.078318+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:25:29.602198+0000","scrub_duration":0.0042055560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"132'2","reported_seq":146,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:27.564836+0000","last_change":"2024-09-13T05:04:27.564916+0000","last_active":"2024-09-13T05:04:27.564836+0000","last_peered":"2024-09-13T05:04:27.564836+0000","last_clean":"2024-09-13T05:04:27.564836+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:27.564836+0000","last_undegraded":"2024-09-13T05:04:27.564836+0000","last_fullsized":"2024-09-13T05:04:27.564836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'2","last_scrub_stamp":"2024-09-13T05:04:27.564817+0000","last_deep_scrub":"132'2","last_deep_scrub_stamp":"2024-09-13T05:04:27.564817+0000","last_clean_scrub_stamp":"2024-09-13T05:04:27.564817+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:46:25.023841+0000","scrub_duration":0.0032450019999999999,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"132'627","reported_seq":4256,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:40.520316+0000","last_change":"2024-09-13T05:04:40.520443+0000","last_active":"2024-09-13T05:04:40.520316+0000","last_peered":"2024-09-13T05:04:40.520316+0000","last_clean":"2024-09-13T05:04:40.520316+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:40.520316+0000","last_undegraded":"2024-09-13T05:04:40.520316+0000","last_fullsized":"2024-09-13T05:04:40.520316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'627","last_scrub_stamp":"2024-09-13T05:04:40.520292+0000","last_deep_scrub":"132'627","last_deep_scrub_stamp":"2024-09-13T05:04:40.520292+0000","last_clean_scrub_stamp":"2024-09-13T05:04:40.520292+0000","objects_scrubbed":19,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:51:35.622564+0000","scrub_duration":0.016126941999999998,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":86,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.989530+0000","last_change":"2024-09-13T05:04:16.988998+0000","last_active":"2024-09-13T05:04:16.989530+0000","last_peered":"2024-09-13T05:04:16.989530+0000","last_clean":"2024-09-13T05:04:16.989530+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:16.989530+0000","last_undegraded":"2024-09-13T05:04:16.989530+0000","last_fullsized":"2024-09-13T05:04:16.989530+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:01:56.162205+0000","last_clean_scrub_stamp":"2024-09-13T05:01:56.162205+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"109'611","reported_seq":3701,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:16.991852+0000","last_change":"2024-09-13T05:04:16.991471+0000","last_active":"2024-09-13T05:04:16.991852+0000","last_peered":"2024-09-13T05:04:16.991852+0000","last_clean":"2024-09-13T05:04:16.991852+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:16.991852+0000","last_undegraded":"2024-09-13T05:04:16.991852+0000","last_fullsized":"2024-09-13T05:04:16.991852+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":611,"log_dups_size":0,"ondisk_log_size":611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":143,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:30.333640+0000","last_change":"2024-09-13T05:04:30.333750+0000","last_active":"2024-09-13T05:04:30.333640+0000","last_peered":"2024-09-13T05:04:30.333640+0000","last_clean":"2024-09-13T05:04:30.333640+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:30.333640+0000","last_undegraded":"2024-09-13T05:04:30.333640+0000","last_fullsized":"2024-09-13T05:04:30.333640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:30.333616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:30.333616+0000","last_clean_scrub_stamp":"2024-09-13T05:04:30.333616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:13:23.974750+0000","scrub_duration":0.001719145,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"132'1194","reported_seq":3939,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:36.002937+0000","last_change":"2024-09-13T05:04:36.003059+0000","last_active":"2024-09-13T05:04:36.002937+0000","last_peered":"2024-09-13T05:04:36.002937+0000","last_clean":"2024-09-13T05:04:36.002937+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:36.002937+0000","last_undegraded":"2024-09-13T05:04:36.002937+0000","last_fullsized":"2024-09-13T05:04:36.002937+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1194","last_scrub_stamp":"2024-09-13T05:04:36.002913+0000","last_deep_scrub":"132'1194","last_deep_scrub_stamp":"2024-09-13T05:04:36.002913+0000","last_clean_scrub_stamp":"2024-09-13T05:04:36.002913+0000","objects_scrubbed":14,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:40.733133+0000","scrub_duration":0.011374762,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:32.923809+0000","last_change":"2024-09-13T05:04:32.923909+0000","last_active":"2024-09-13T05:04:32.923809+0000","last_peered":"2024-09-13T05:04:32.923809+0000","last_clean":"2024-09-13T05:04:32.923809+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:32.923809+0000","last_undegraded":"2024-09-13T05:04:32.923809+0000","last_fullsized":"2024-09-13T05:04:32.923809+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:32.923787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:32.923787+0000","last_clean_scrub_stamp":"2024-09-13T05:04:32.923787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:56:57.913618+0000","scrub_duration":0.001859943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"132'454","reported_seq":3753,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:40.059449+0000","last_change":"2024-09-13T05:04:40.059573+0000","last_active":"2024-09-13T05:04:40.059449+0000","last_peered":"2024-09-13T05:04:40.059449+0000","last_clean":"2024-09-13T05:04:40.059449+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:40.059449+0000","last_undegraded":"2024-09-13T05:04:40.059449+0000","last_fullsized":"2024-09-13T05:04:40.059449+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'454","last_scrub_stamp":"2024-09-13T05:04:40.059426+0000","last_deep_scrub":"132'454","last_deep_scrub_stamp":"2024-09-13T05:04:40.059426+0000","last_clean_scrub_stamp":"2024-09-13T05:04:40.059426+0000","objects_scrubbed":8,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:59:39.040897+0000","scrub_duration":0.0073298180000000001,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"132'470","reported_seq":2894,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:37.339240+0000","last_change":"2024-09-13T05:04:37.339364+0000","last_active":"2024-09-13T05:04:37.339240+0000","last_peered":"2024-09-13T05:04:37.339240+0000","last_clean":"2024-09-13T05:04:37.339240+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:37.339240+0000","last_undegraded":"2024-09-13T05:04:37.339240+0000","last_fullsized":"2024-09-13T05:04:37.339240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'470","last_scrub_stamp":"2024-09-13T05:04:37.339217+0000","last_deep_scrub":"132'470","last_deep_scrub_stamp":"2024-09-13T05:04:37.339217+0000","last_clean_scrub_stamp":"2024-09-13T05:04:37.339217+0000","objects_scrubbed":7,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:44.880836+0000","scrub_duration":0.0069644119999999997,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'660","reported_seq":4688,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:32.942455+0000","last_change":"2024-09-13T05:04:32.942588+0000","last_active":"2024-09-13T05:04:32.942455+0000","last_peered":"2024-09-13T05:04:32.942455+0000","last_clean":"2024-09-13T05:04:32.942455+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:32.942455+0000","last_undegraded":"2024-09-13T05:04:32.942455+0000","last_fullsized":"2024-09-13T05:04:32.942455+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'660","last_scrub_stamp":"2024-09-13T05:04:32.942433+0000","last_deep_scrub":"132'660","last_deep_scrub_stamp":"2024-09-13T05:04:32.942433+0000","last_clean_scrub_stamp":"2024-09-13T05:04:32.942433+0000","objects_scrubbed":17,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:17.553587+0000","scrub_duration":0.013735855999999999,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"132'894","reported_seq":2849,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:30.987588+0000","last_change":"2024-09-13T05:04:30.987717+0000","last_active":"2024-09-13T05:04:30.987588+0000","last_peered":"2024-09-13T05:04:30.987588+0000","last_clean":"2024-09-13T05:04:30.987588+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:30.987588+0000","last_undegraded":"2024-09-13T05:04:30.987588+0000","last_fullsized":"2024-09-13T05:04:30.987588+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'894","last_scrub_stamp":"2024-09-13T05:04:30.987564+0000","last_deep_scrub":"132'894","last_deep_scrub_stamp":"2024-09-13T05:04:30.987564+0000","last_clean_scrub_stamp":"2024-09-13T05:04:30.987564+0000","objects_scrubbed":9,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:45.863587+0000","scrub_duration":0.0086435769999999995,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"132'449","reported_seq":3826,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:45.090968+0000","last_change":"2024-09-13T05:04:45.091086+0000","last_active":"2024-09-13T05:04:45.090968+0000","last_peered":"2024-09-13T05:04:45.090968+0000","last_clean":"2024-09-13T05:04:45.090968+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:45.090968+0000","last_undegraded":"2024-09-13T05:04:45.090968+0000","last_fullsized":"2024-09-13T05:04:45.090968+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'449","last_scrub_stamp":"2024-09-13T05:04:45.090930+0000","last_deep_scrub":"132'449","last_deep_scrub_stamp":"2024-09-13T05:04:45.090930+0000","last_clean_scrub_stamp":"2024-09-13T05:04:45.090930+0000","objects_scrubbed":12,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:37.485192+0000","scrub_duration":0.0089008170000000001,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"106'570","reported_seq":5293,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:21.120585+0000","last_change":"2024-09-13T05:04:16.985647+0000","last_active":"2024-09-13T05:04:21.120585+0000","last_peered":"2024-09-13T05:04:21.120585+0000","last_clean":"2024-09-13T05:04:21.120585+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:04:21.120585+0000","last_undegraded":"2024-09-13T05:04:21.120585+0000","last_fullsized":"2024-09-13T05:04:21.120585+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"132'589","reported_seq":5637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:45.966973+0000","last_change":"2024-09-13T05:04:45.967134+0000","last_active":"2024-09-13T05:04:45.966973+0000","last_peered":"2024-09-13T05:04:45.966973+0000","last_clean":"2024-09-13T05:04:45.966973+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:45.966973+0000","last_undegraded":"2024-09-13T05:04:45.966973+0000","last_fullsized":"2024-09-13T05:04:45.966973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'589","last_scrub_stamp":"2024-09-13T05:04:45.966929+0000","last_deep_scrub":"132'589","last_deep_scrub_stamp":"2024-09-13T05:04:45.966929+0000","last_clean_scrub_stamp":"2024-09-13T05:04:45.966929+0000","objects_scrubbed":14,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:04.494166+0000","scrub_duration":0.011784565,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"132'399","reported_seq":3057,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.057065+0000","last_change":"2024-09-13T05:04:38.057170+0000","last_active":"2024-09-13T05:04:38.057065+0000","last_peered":"2024-09-13T05:04:38.057065+0000","last_clean":"2024-09-13T05:04:38.057065+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:38.057065+0000","last_undegraded":"2024-09-13T05:04:38.057065+0000","last_fullsized":"2024-09-13T05:04:38.057065+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'399","last_scrub_stamp":"2024-09-13T05:04:38.057039+0000","last_deep_scrub":"132'399","last_deep_scrub_stamp":"2024-09-13T05:04:38.057039+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.057039+0000","objects_scrubbed":9,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:45.966737+0000","scrub_duration":0.0072941580000000002,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"132'886","reported_seq":3127,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:31.930985+0000","last_change":"2024-09-13T05:04:31.931128+0000","last_active":"2024-09-13T05:04:31.930985+0000","last_peered":"2024-09-13T05:04:31.930985+0000","last_clean":"2024-09-13T05:04:31.930985+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:31.930985+0000","last_undegraded":"2024-09-13T05:04:31.930985+0000","last_fullsized":"2024-09-13T05:04:31.930985+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'886","last_scrub_stamp":"2024-09-13T05:04:31.930960+0000","last_deep_scrub":"132'886","last_deep_scrub_stamp":"2024-09-13T05:04:31.930960+0000","last_clean_scrub_stamp":"2024-09-13T05:04:31.930960+0000","objects_scrubbed":12,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:12:22.292092+0000","scrub_duration":0.010054027,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"132'559","reported_seq":4262,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:34.957996+0000","last_change":"2024-09-13T05:04:34.958114+0000","last_active":"2024-09-13T05:04:34.957996+0000","last_peered":"2024-09-13T05:04:34.957996+0000","last_clean":"2024-09-13T05:04:34.957996+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:34.957996+0000","last_undegraded":"2024-09-13T05:04:34.957996+0000","last_fullsized":"2024-09-13T05:04:34.957996+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'559","last_scrub_stamp":"2024-09-13T05:04:34.957972+0000","last_deep_scrub":"132'559","last_deep_scrub_stamp":"2024-09-13T05:04:34.957972+0000","last_clean_scrub_stamp":"2024-09-13T05:04:34.957972+0000","objects_scrubbed":8,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:14:42.577985+0000","scrub_duration":0.0076218420000000002,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"108'465","reported_seq":3847,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:26.123000+0000","last_change":"2024-09-13T05:03:52.983562+0000","last_active":"2024-09-13T05:04:26.123000+0000","last_peered":"2024-09-13T05:04:26.123000+0000","last_clean":"2024-09-13T05:04:26.123000+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:26.123000+0000","last_undegraded":"2024-09-13T05:04:26.123000+0000","last_fullsized":"2024-09-13T05:04:26.123000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:02:05.864496+0000","last_clean_scrub_stamp":"2024-09-13T05:02:05.864496+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"132'771","reported_seq":3303,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:29.965841+0000","last_change":"2024-09-13T05:04:29.965960+0000","last_active":"2024-09-13T05:04:29.965841+0000","last_peered":"2024-09-13T05:04:29.965841+0000","last_clean":"2024-09-13T05:04:29.965841+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:29.965841+0000","last_undegraded":"2024-09-13T05:04:29.965841+0000","last_fullsized":"2024-09-13T05:04:29.965841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'771","last_scrub_stamp":"2024-09-13T05:04:29.965819+0000","last_deep_scrub":"132'771","last_deep_scrub_stamp":"2024-09-13T05:04:29.965819+0000","last_clean_scrub_stamp":"2024-09-13T05:04:29.965819+0000","objects_scrubbed":10,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:25:52.349432+0000","scrub_duration":0.0094627819999999994,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20392,"ondisk_log_size":20392,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312884,"statfs":{"total":96636764160,"available":96576393216,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.45800000000000002,"15min":0.46899999999999997},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56899999999999995,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.436,"15min":0.442},"min":{"1min":0.23999999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.55100000000000005,"5min":0.622,"15min":0.622},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.44900000000000001,"15min":0.45800000000000002},"min":{"1min":0.307,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.53400000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.46500000000000002,"15min":0.47299999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.53600000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.51700000000000002,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.495,"15min":0.505},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.53300000000000003,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.61899999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58999999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48399999999999999,"15min":0.48799999999999999},"min":{"1min":0.38200000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.57399999999999995,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:04:02 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.247,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.27100000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.68899999999999995,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.38800000000000001,"15min":0.39100000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.54900000000000004,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.28899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.57199999999999995,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.501}]},{"osd":6,"last update":"Fri Sep 13 05:03:56 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.44700000000000001,"15min":0.45100000000000001},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59499999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.432,"5min":0.46000000000000002,"15min":0.46400000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.623,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.58599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40848,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330992,"statfs":{"total":96636764160,"available":96594935808,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.39700000000000002,"15min":0.39200000000000002},"min":{"1min":0.28999999999999998,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.46100000000000002,"15min":0.46300000000000002},"min":{"1min":0.27300000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.53900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.53200000000000003}]},{"osd":1,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.41399999999999998,"15min":0.40799999999999997},"min":{"1min":0.28799999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.315,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.57999999999999996,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.443,"15min":0.443},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56799999999999995,"5min":0.625,"15min":0.625},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45000000000000001,"15min":0.44800000000000001},"min":{"1min":0.34000000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.61799999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.54600000000000004}]},{"osd":3,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.441,"15min":0.441},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.625,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.47499999999999998,"15min":0.47899999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.57499999999999996,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.36699999999999999,"15min":0.36499999999999999},"min":{"1min":0.252,"5min":0.224,"15min":0.224},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68200000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.51300000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40500000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:04:01 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.315,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.32300000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.55600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54800,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317040,"statfs":{"total":96636764160,"available":96580648960,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.41599999999999998,"15min":0.41199999999999998},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.60199999999999998,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.41599999999999998},"min":{"1min":0.27700000000000002,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.64500000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.44600000000000001},"min":{"1min":0.33400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.44600000000000001,"15min":0.44400000000000001},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.67000000000000004,"5min":0.875,"15min":0.875},"last":0.61799999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.434,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.47299999999999998,"15min":0.46899999999999997},"min":{"1min":0.36799999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.69599999999999995}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.45100000000000001,"15min":0.44500000000000001},"min":{"1min":0.34399999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.74299999999999999}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39900000000000002,"15min":0.40300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53600000000000003,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.623,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.44400000000000001,"15min":0.45000000000000001},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.73099999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.44900000000000001,"15min":0.45300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71899999999999997,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55400000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47799999999999998,"15min":0.48399999999999999},"min":{"1min":0.33700000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.68700000000000006,"5min":1.006,"15min":1.006},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.5,"15min":0.502},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75700000000000001,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68756,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303084,"statfs":{"total":96636764160,"available":96566358016,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.71199999999999997,"15min":0.75800000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.58099999999999996,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.69299999999999995,"15min":0.73099999999999998},"min":{"1min":0.23300000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.68799999999999994,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.70899999999999996,"5min":2.4049999999999998,"15min":2.4049999999999998},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.71599999999999997,"15min":0.754},"min":{"1min":0.32600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.76000000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.30299999999999999,"5min":0.247,"15min":0.247},"max":{"1min":0.65500000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.75600000000000001,"15min":0.79700000000000004},"min":{"1min":0.377,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73299999999999998,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.64500000000000002,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.68200000000000005,"15min":0.71499999999999997},"min":{"1min":0.309,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.67500000000000004,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.65000000000000002,"15min":0.69499999999999995},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64700000000000002,"5min":6.577,"15min":6.577},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.53300000000000003,"15min":0.55100000000000005},"min":{"1min":0.27800000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.63100000000000001,"5min":3.379,"15min":3.379},"last":0.377}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.28399999999999997,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.58499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.442,"5min":0.53000000000000003,"15min":0.54400000000000004},"min":{"1min":0.32300000000000001,"5min":0.255,"15min":0.255},"max":{"1min":0.72199999999999998,"5min":2.371,"15min":2.371},"last":0.46800000000000003}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.71199999999999997,"15min":0.752},"min":{"1min":0.29599999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.65500000000000003,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.28299999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.45100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68752,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303088,"statfs":{"total":96636764160,"available":96566362112,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.33900000000000002,"15min":0.33000000000000002},"min":{"1min":0.215,"5min":0.19,"15min":0.19},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.38300000000000001,"15min":0.375},"min":{"1min":0.23599999999999999,"5min":0.224,"15min":0.224},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.29899999999999999}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.36399999999999999,"15min":0.35799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.40400000000000003,"15min":0.38700000000000001},"min":{"1min":0.25900000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.40600000000000003,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.44,"15min":0.42599999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.372}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.32600000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.57999999999999996}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57199999999999995,"15min":0.57999999999999996},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.90300000000000002,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.625,"15min":0.623},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.8360000000000001,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.772,"5min":1.772,"15min":1.772},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.83999999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.436}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.38900000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90300000000000002,"5min":1.222,"15min":1.222},"last":0.505},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.67200000000000004,"15min":0.68200000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.5820000000000001,"5min":3.02,"15min":3.02},"last":0.47299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343280,"statfs":{"total":96636764160,"available":96607518720,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.36399999999999999,"15min":0.36799999999999999},"min":{"1min":0.224,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52000000000000002,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.498},{"interface":"front","average":{"1min":0.32400000000000001,"5min":0.36799999999999999,"15min":0.375},"min":{"1min":0.251,"5min":0.218,"15min":0.218},"max":{"1min":0.45700000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.36099999999999999,"15min":0.36699999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.46999999999999997,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.442},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.26000000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57899999999999996,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.53200000000000003}]},{"osd":3,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.39700000000000002,"15min":0.40300000000000002},"min":{"1min":0.28499999999999998,"5min":0.249,"15min":0.249},"max":{"1min":0.45100000000000001,"5min":1,"15min":1},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.47999999999999998,"5min":1.109,"15min":1.109},"last":0.55900000000000005}]},{"osd":4,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.53900000000000003,"15min":0.54600000000000004},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76300000000000001,"5min":1.046,"15min":1.046},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.60699999999999998,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.72399999999999998,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.73699999999999999,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.80600000000000005}]},{"osd":6,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.39600000000000002,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67100000000000004,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.5,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.68999999999999995,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.82499999999999996}]},{"osd":7,"last update":"Fri Sep 13 05:03:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.39800000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.72599999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.85399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72908,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298932,"statfs":{"total":96636764160,"available":96562106368,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34300000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.311}]},{"osd":2,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32400000000000001,"15min":0.32200000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.38900000000000001,"15min":0.39500000000000002},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.57699999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41699999999999998}]},{"osd":3,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42999999999999999,"15min":0.432},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.59299999999999997,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.28199999999999997,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.53700000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.29299999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.495,"15min":0.49399999999999999},"min":{"1min":0.35699999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76600000000000001,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51100000000000001,"15min":0.50900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69899999999999995,"5min":0.87,"15min":0.87},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.435},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77400000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53300000000000003,"15min":0.53500000000000003},"min":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79900000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76700000000000002,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.496}]},{"osd":7,"last update":"Fri Sep 13 05:03:58 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55700000000000005,"15min":0.55900000000000005},"min":{"1min":0.42399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.70899999999999996,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.56999999999999995,"15min":0.57199999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58900,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312940,"statfs":{"total":96636764160,"available":96576450560,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.45200000000000001,"15min":0.36699999999999999},"min":{"1min":0.245,"5min":0.184,"15min":0.184},"max":{"1min":11.625,"5min":11.625,"15min":11.625},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.41199999999999998,"15min":0.38},"min":{"1min":0.29199999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":4.9660000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.36799999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.42399999999999999,"15min":0.374},"min":{"1min":0.26000000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.2080000000000002,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.36899999999999999,"15min":0.36199999999999999},"min":{"1min":0.26700000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.307}]},{"osd":3,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.833,"5min":1.833,"15min":1.833},"last":0.377},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.499,"15min":0.44500000000000001},"min":{"1min":0.315,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":8.0340000000000007,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.40100000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.501,"15min":0.49299999999999999},"min":{"1min":0.34799999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.56299999999999994,"15min":0.51000000000000001},"min":{"1min":0.377,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":7.2939999999999996,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.432}]},{"osd":5,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.55500000000000005,"15min":0.53200000000000003},"min":{"1min":0.36399999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.7229999999999999,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.56299999999999994,"15min":0.54200000000000004},"min":{"1min":0.42299999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.54600000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.64300000000000002,"15min":0.57999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":9.1579999999999995,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.51200000000000001},{"interface":"front","average":{"1min":1.0429999999999999,"5min":0.66300000000000003,"15min":0.59899999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":9.1929999999999996,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.61799999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:04:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.38300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.60099999999999998,"15min":0.56000000000000005},"min":{"1min":0.40300000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":6.3730000000000002,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.499}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:04:50.516 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.516 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.516 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.516 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.516 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.517 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.517 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-13T05:01:56.162205+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=1, tm_sec=56, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.517 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.517 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.517 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-13T05:02:05.864496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=2, tm_sec=5, tm_wday=4, tm_yday=257, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=13, tm_hour=5, tm_min=4, tm_sec=25, tm_wday=4, tm_yday=257, tm_isdst=0) 2024-09-13T05:04:50.517 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-13T05:05:10.518 DEBUG:teuthology.orchestra.run.smithi053:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-13T05:05:10.818 INFO:teuthology.orchestra.run.smithi053.stdout: 2024-09-13T05:05:10.819 INFO:teuthology.orchestra.run.smithi053.stderr:dumped all 2024-09-13T05:05:10.830 INFO:teuthology.orchestra.run.smithi053.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-13T05:05:09.975504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":343,"num_object_clones":0,"num_object_copies":686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":343,"num_whiteouts":335,"num_read":148249,"num_read_kb":707103,"num_write":23897,"num_write_kb":384104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20609,"ondisk_log_size":20609,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":456576,"kb_used_data":12480,"kb_used_omap":12,"kb_used_meta":444019,"kb_avail":754518144,"statfs":{"total":773094113280,"available":772626579456,"internally_reserved":0,"allocated":12779520,"data_stored":7239004,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12743,"internal_metadata":454676025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"3.1f","version":"132'372","reported_seq":2611,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:05:04.132542+0000","last_change":"2024-09-13T05:05:04.132713+0000","last_active":"2024-09-13T05:05:04.132542+0000","last_peered":"2024-09-13T05:05:04.132542+0000","last_clean":"2024-09-13T05:05:04.132542+0000","last_became_active":"2024-09-13T05:02:06.881251+0000","last_became_peered":"2024-09-13T05:02:06.881251+0000","last_unstale":"2024-09-13T05:05:04.132542+0000","last_undegraded":"2024-09-13T05:05:04.132542+0000","last_fullsized":"2024-09-13T05:05:04.132542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'372","last_scrub_stamp":"2024-09-13T05:05:04.132516+0000","last_deep_scrub":"132'372","last_deep_scrub_stamp":"2024-09-13T05:05:04.132516+0000","last_clean_scrub_stamp":"2024-09-13T05:05:04.132516+0000","objects_scrubbed":14,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:59:30.659129+0000","scrub_duration":0.010740223,"objects_trimmed":0,"snaptrim_duration":0.001682834,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":2431,"num_read_kb":12703,"num_write":303,"num_write_kb":12682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"132'562","reported_seq":2906,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:28.957842+0000","last_change":"2024-09-13T05:04:28.957923+0000","last_active":"2024-09-13T05:04:28.957842+0000","last_peered":"2024-09-13T05:04:28.957842+0000","last_clean":"2024-09-13T05:04:28.957842+0000","last_became_active":"2024-09-13T05:04:03.917679+0000","last_became_peered":"2024-09-13T05:04:03.917679+0000","last_unstale":"2024-09-13T05:04:28.957842+0000","last_undegraded":"2024-09-13T05:04:28.957842+0000","last_fullsized":"2024-09-13T05:04:28.957842+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'562","last_scrub_stamp":"2024-09-13T05:04:28.957821+0000","last_deep_scrub":"132'562","last_deep_scrub_stamp":"2024-09-13T05:04:28.957821+0000","last_clean_scrub_stamp":"2024-09-13T05:04:28.957821+0000","objects_scrubbed":7,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:54:57.988677+0000","scrub_duration":0.0067411629999999997,"objects_trimmed":0,"snaptrim_duration":0.001853822,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3165,"num_read_kb":18952,"num_write":648,"num_write_kb":17872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"132'519","reported_seq":2686,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:51.998648+0000","last_change":"2024-09-13T05:04:51.998774+0000","last_active":"2024-09-13T05:04:51.998648+0000","last_peered":"2024-09-13T05:04:51.998648+0000","last_clean":"2024-09-13T05:04:51.998648+0000","last_became_active":"2024-09-13T05:02:06.883791+0000","last_became_peered":"2024-09-13T05:02:06.883791+0000","last_unstale":"2024-09-13T05:04:51.998648+0000","last_undegraded":"2024-09-13T05:04:51.998648+0000","last_fullsized":"2024-09-13T05:04:51.998648+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'519","last_scrub_stamp":"2024-09-13T05:04:51.998624+0000","last_deep_scrub":"132'519","last_deep_scrub_stamp":"2024-09-13T05:04:51.998624+0000","last_clean_scrub_stamp":"2024-09-13T05:04:51.998624+0000","objects_scrubbed":8,"log_size":519,"log_dups_size":0,"ondisk_log_size":519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:30:30.184805+0000","scrub_duration":0.0067096070000000002,"objects_trimmed":0,"snaptrim_duration":0.00097320499999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3851,"num_read_kb":7744,"num_write":588,"num_write_kb":3105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"132'609","reported_seq":5517,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:34.250565+0000","last_change":"2024-09-13T05:04:34.250565+0000","last_active":"2024-09-13T05:04:34.250565+0000","last_peered":"2024-09-13T05:04:34.250565+0000","last_clean":"2024-09-13T05:04:34.250565+0000","last_became_active":"2024-09-13T05:02:06.882439+0000","last_became_peered":"2024-09-13T05:02:06.882439+0000","last_unstale":"2024-09-13T05:04:34.250565+0000","last_undegraded":"2024-09-13T05:04:34.250565+0000","last_fullsized":"2024-09-13T05:04:34.250565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'609","last_scrub_stamp":"2024-09-13T05:04:34.250433+0000","last_deep_scrub":"132'609","last_deep_scrub_stamp":"2024-09-13T05:04:34.250433+0000","last_clean_scrub_stamp":"2024-09-13T05:04:34.250433+0000","objects_scrubbed":10,"log_size":609,"log_dups_size":0,"ondisk_log_size":609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:25:18.749268+0000","scrub_duration":0.0089988020000000002,"objects_trimmed":0,"snaptrim_duration":0.0012258690000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":6356,"num_read_kb":35086,"num_write":723,"num_write_kb":21063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"132'744","reported_seq":5574,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:56.106413+0000","last_change":"2024-09-13T05:04:56.106529+0000","last_active":"2024-09-13T05:04:56.106413+0000","last_peered":"2024-09-13T05:04:56.106413+0000","last_clean":"2024-09-13T05:04:56.106413+0000","last_became_active":"2024-09-13T05:02:06.877747+0000","last_became_peered":"2024-09-13T05:02:06.877747+0000","last_unstale":"2024-09-13T05:04:56.106413+0000","last_undegraded":"2024-09-13T05:04:56.106413+0000","last_fullsized":"2024-09-13T05:04:56.106413+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'744","last_scrub_stamp":"2024-09-13T05:04:56.106388+0000","last_deep_scrub":"132'744","last_deep_scrub_stamp":"2024-09-13T05:04:56.106388+0000","last_clean_scrub_stamp":"2024-09-13T05:04:56.106388+0000","objects_scrubbed":15,"log_size":744,"log_dups_size":0,"ondisk_log_size":744,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:08:06.631548+0000","scrub_duration":0.012132575,"objects_trimmed":0,"snaptrim_duration":0.001901029,"stat_sum":{"num_bytes":0,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":15,"num_read":7592,"num_read_kb":27519,"num_write":697,"num_write_kb":3895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"132'1816","reported_seq":6404,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:41.417700+0000","last_change":"2024-09-13T05:04:41.417831+0000","last_active":"2024-09-13T05:04:41.417700+0000","last_peered":"2024-09-13T05:04:41.417700+0000","last_clean":"2024-09-13T05:04:41.417700+0000","last_became_active":"2024-09-13T05:02:06.880975+0000","last_became_peered":"2024-09-13T05:02:06.880975+0000","last_unstale":"2024-09-13T05:04:41.417700+0000","last_undegraded":"2024-09-13T05:04:41.417700+0000","last_fullsized":"2024-09-13T05:04:41.417700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1816","last_scrub_stamp":"2024-09-13T05:04:41.417678+0000","last_deep_scrub":"132'1816","last_deep_scrub_stamp":"2024-09-13T05:04:41.417678+0000","last_clean_scrub_stamp":"2024-09-13T05:04:41.417678+0000","objects_scrubbed":15,"log_size":1816,"log_dups_size":0,"ondisk_log_size":1816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:49:01.745357+0000","scrub_duration":0.0149617,"objects_trimmed":0,"snaptrim_duration":0.00056116499999999997,"stat_sum":{"num_bytes":19,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":14,"num_read":12394,"num_read_kb":32143,"num_write":2884,"num_write_kb":15674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"132'471","reported_seq":2164,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:33.480073+0000","last_change":"2024-09-13T05:04:33.480186+0000","last_active":"2024-09-13T05:04:33.480073+0000","last_peered":"2024-09-13T05:04:33.480073+0000","last_clean":"2024-09-13T05:04:33.480073+0000","last_became_active":"2024-09-13T05:04:03.917555+0000","last_became_peered":"2024-09-13T05:04:03.917555+0000","last_unstale":"2024-09-13T05:04:33.480073+0000","last_undegraded":"2024-09-13T05:04:33.480073+0000","last_fullsized":"2024-09-13T05:04:33.480073+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'471","last_scrub_stamp":"2024-09-13T05:04:33.480048+0000","last_deep_scrub":"132'471","last_deep_scrub_stamp":"2024-09-13T05:04:33.480048+0000","last_clean_scrub_stamp":"2024-09-13T05:04:33.480048+0000","objects_scrubbed":8,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:34:02.181935+0000","scrub_duration":0.0073536670000000004,"objects_trimmed":0,"snaptrim_duration":0.00048031099999999998,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":2464,"num_read_kb":10969,"num_write":531,"num_write_kb":6696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"132'408","reported_seq":2700,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:33.055703+0000","last_change":"2024-09-13T05:04:33.055855+0000","last_active":"2024-09-13T05:04:33.055703+0000","last_peered":"2024-09-13T05:04:33.055703+0000","last_clean":"2024-09-13T05:04:33.055703+0000","last_became_active":"2024-09-13T05:04:03.917987+0000","last_became_peered":"2024-09-13T05:04:03.917987+0000","last_unstale":"2024-09-13T05:04:33.055703+0000","last_undegraded":"2024-09-13T05:04:33.055703+0000","last_fullsized":"2024-09-13T05:04:33.055703+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'408","last_scrub_stamp":"2024-09-13T05:04:33.055680+0000","last_deep_scrub":"132'408","last_deep_scrub_stamp":"2024-09-13T05:04:33.055680+0000","last_clean_scrub_stamp":"2024-09-13T05:04:33.055680+0000","objects_scrubbed":9,"log_size":408,"log_dups_size":0,"ondisk_log_size":408,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:06:43.875417+0000","scrub_duration":0.0083631799999999996,"objects_trimmed":0,"snaptrim_duration":0.00095383200000000005,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2298,"num_read_kb":13275,"num_write":480,"num_write_kb":21708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"132'393","reported_seq":2811,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:47.189897+0000","last_change":"2024-09-13T05:04:47.190031+0000","last_active":"2024-09-13T05:04:47.189897+0000","last_peered":"2024-09-13T05:04:47.189897+0000","last_clean":"2024-09-13T05:04:47.189897+0000","last_became_active":"2024-09-13T05:02:06.880696+0000","last_became_peered":"2024-09-13T05:02:06.880696+0000","last_unstale":"2024-09-13T05:04:47.189897+0000","last_undegraded":"2024-09-13T05:04:47.189897+0000","last_fullsized":"2024-09-13T05:04:47.189897+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'393","last_scrub_stamp":"2024-09-13T05:04:47.189874+0000","last_deep_scrub":"132'393","last_deep_scrub_stamp":"2024-09-13T05:04:47.189874+0000","last_clean_scrub_stamp":"2024-09-13T05:04:47.189874+0000","objects_scrubbed":4,"log_size":393,"log_dups_size":0,"ondisk_log_size":393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:31:52.295552+0000","scrub_duration":0.0052053120000000001,"objects_trimmed":0,"snaptrim_duration":0.0015866579999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2745,"num_read_kb":13827,"num_write":380,"num_write_kb":7231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"132'367","reported_seq":3297,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:36.280067+0000","last_change":"2024-09-13T05:04:36.280174+0000","last_active":"2024-09-13T05:04:36.280067+0000","last_peered":"2024-09-13T05:04:36.280067+0000","last_clean":"2024-09-13T05:04:36.280067+0000","last_became_active":"2024-09-13T05:02:06.885332+0000","last_became_peered":"2024-09-13T05:02:06.885332+0000","last_unstale":"2024-09-13T05:04:36.280067+0000","last_undegraded":"2024-09-13T05:04:36.280067+0000","last_fullsized":"2024-09-13T05:04:36.280067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'367","last_scrub_stamp":"2024-09-13T05:04:36.280044+0000","last_deep_scrub":"132'367","last_deep_scrub_stamp":"2024-09-13T05:04:36.280044+0000","last_clean_scrub_stamp":"2024-09-13T05:04:36.280044+0000","objects_scrubbed":7,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:01:29.404235+0000","scrub_duration":0.0063987940000000002,"objects_trimmed":0,"snaptrim_duration":0.000690656,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":3239,"num_read_kb":17273,"num_write":315,"num_write_kb":5905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"132'647","reported_seq":3661,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:05:01.853020+0000","last_change":"2024-09-13T05:05:01.853136+0000","last_active":"2024-09-13T05:05:01.853020+0000","last_peered":"2024-09-13T05:05:01.853020+0000","last_clean":"2024-09-13T05:05:01.853020+0000","last_became_active":"2024-09-13T05:04:16.989020+0000","last_became_peered":"2024-09-13T05:04:16.989020+0000","last_unstale":"2024-09-13T05:05:01.853020+0000","last_undegraded":"2024-09-13T05:05:01.853020+0000","last_fullsized":"2024-09-13T05:05:01.853020+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'647","last_scrub_stamp":"2024-09-13T05:05:01.852998+0000","last_deep_scrub":"132'647","last_deep_scrub_stamp":"2024-09-13T05:05:01.852998+0000","last_clean_scrub_stamp":"2024-09-13T05:05:01.852998+0000","objects_scrubbed":11,"log_size":647,"log_dups_size":0,"ondisk_log_size":647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:22:41.898924+0000","scrub_duration":0.0089432780000000007,"objects_trimmed":1,"snaptrim_duration":0.003414266,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4194,"num_read_kb":19459,"num_write":840,"num_write_kb":23405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"132'355","reported_seq":2158,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.359014+0000","last_change":"2024-09-13T05:04:38.359143+0000","last_active":"2024-09-13T05:04:38.359014+0000","last_peered":"2024-09-13T05:04:38.359014+0000","last_clean":"2024-09-13T05:04:38.359014+0000","last_became_active":"2024-09-13T05:02:06.885386+0000","last_became_peered":"2024-09-13T05:02:06.885386+0000","last_unstale":"2024-09-13T05:04:38.359014+0000","last_undegraded":"2024-09-13T05:04:38.359014+0000","last_fullsized":"2024-09-13T05:04:38.359014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'355","last_scrub_stamp":"2024-09-13T05:04:38.358982+0000","last_deep_scrub":"132'355","last_deep_scrub_stamp":"2024-09-13T05:04:38.358982+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.358982+0000","objects_scrubbed":7,"log_size":355,"log_dups_size":0,"ondisk_log_size":355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:55:12.706645+0000","scrub_duration":0.0064214700000000003,"objects_trimmed":0,"snaptrim_duration":0.00086267500000000003,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":1615,"num_read_kb":13795,"num_write":326,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.094922+0000","last_change":"2024-09-13T05:04:38.095070+0000","last_active":"2024-09-13T05:04:38.094922+0000","last_peered":"2024-09-13T05:04:38.094922+0000","last_clean":"2024-09-13T05:04:38.094922+0000","last_became_active":"2024-09-13T05:04:14.969773+0000","last_became_peered":"2024-09-13T05:04:14.969773+0000","last_unstale":"2024-09-13T05:04:38.094922+0000","last_undegraded":"2024-09-13T05:04:38.094922+0000","last_fullsized":"2024-09-13T05:04:38.094922+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:38.094902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:38.094902+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.094902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:23:53.100613+0000","scrub_duration":0.001682458,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"132'479","reported_seq":3207,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.251889+0000","last_change":"2024-09-13T05:04:38.252027+0000","last_active":"2024-09-13T05:04:38.251889+0000","last_peered":"2024-09-13T05:04:38.251889+0000","last_clean":"2024-09-13T05:04:38.251889+0000","last_became_active":"2024-09-13T05:04:04.921220+0000","last_became_peered":"2024-09-13T05:04:04.921220+0000","last_unstale":"2024-09-13T05:04:38.251889+0000","last_undegraded":"2024-09-13T05:04:38.251889+0000","last_fullsized":"2024-09-13T05:04:38.251889+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'479","last_scrub_stamp":"2024-09-13T05:04:38.251866+0000","last_deep_scrub":"132'479","last_deep_scrub_stamp":"2024-09-13T05:04:38.251866+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.251866+0000","objects_scrubbed":11,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:57:02.597436+0000","scrub_duration":0.0094311050000000004,"objects_trimmed":0,"snaptrim_duration":0.001384024,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2575,"num_read_kb":11154,"num_write":414,"num_write_kb":3717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:40.074762+0000","last_change":"2024-09-13T05:04:40.074869+0000","last_active":"2024-09-13T05:04:40.074762+0000","last_peered":"2024-09-13T05:04:40.074762+0000","last_clean":"2024-09-13T05:04:40.074762+0000","last_became_active":"2024-09-13T05:04:14.969734+0000","last_became_peered":"2024-09-13T05:04:14.969734+0000","last_unstale":"2024-09-13T05:04:40.074762+0000","last_undegraded":"2024-09-13T05:04:40.074762+0000","last_fullsized":"2024-09-13T05:04:40.074762+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:40.074740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:40.074740+0000","last_clean_scrub_stamp":"2024-09-13T05:04:40.074740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:36:49.156896+0000","scrub_duration":0.0019662630000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"132'1133","reported_seq":5013,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:32.522173+0000","last_change":"2024-09-13T05:04:32.522334+0000","last_active":"2024-09-13T05:04:32.522173+0000","last_peered":"2024-09-13T05:04:32.522173+0000","last_clean":"2024-09-13T05:04:32.522173+0000","last_became_active":"2024-09-13T05:02:06.879751+0000","last_became_peered":"2024-09-13T05:02:06.879751+0000","last_unstale":"2024-09-13T05:04:32.522173+0000","last_undegraded":"2024-09-13T05:04:32.522173+0000","last_fullsized":"2024-09-13T05:04:32.522173+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1133","last_scrub_stamp":"2024-09-13T05:04:32.522147+0000","last_deep_scrub":"132'1133","last_deep_scrub_stamp":"2024-09-13T05:04:32.522147+0000","last_clean_scrub_stamp":"2024-09-13T05:04:32.522147+0000","objects_scrubbed":10,"log_size":1133,"log_dups_size":0,"ondisk_log_size":1133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:30.658913+0000","scrub_duration":0.010768115,"objects_trimmed":0,"snaptrim_duration":0.00066233299999999998,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8632,"num_read_kb":33212,"num_write":1336,"num_write_kb":8933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":144,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:33.399149+0000","last_change":"2024-09-13T05:04:33.399292+0000","last_active":"2024-09-13T05:04:33.399149+0000","last_peered":"2024-09-13T05:04:33.399149+0000","last_clean":"2024-09-13T05:04:33.399149+0000","last_became_active":"2024-09-13T05:01:57.175499+0000","last_became_peered":"2024-09-13T05:01:57.175499+0000","last_unstale":"2024-09-13T05:04:33.399149+0000","last_undegraded":"2024-09-13T05:04:33.399149+0000","last_fullsized":"2024-09-13T05:04:33.399149+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:33.399128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:33.399128+0000","last_clean_scrub_stamp":"2024-09-13T05:04:33.399128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T10:02:47.696263+0000","scrub_duration":0.0021928830000000001,"objects_trimmed":0,"snaptrim_duration":0.00038322399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"132'790","reported_seq":2577,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.010912+0000","last_change":"2024-09-13T05:04:38.011021+0000","last_active":"2024-09-13T05:04:38.010912+0000","last_peered":"2024-09-13T05:04:38.010912+0000","last_clean":"2024-09-13T05:04:38.010912+0000","last_became_active":"2024-09-13T05:04:04.922910+0000","last_became_peered":"2024-09-13T05:04:04.922910+0000","last_unstale":"2024-09-13T05:04:38.010912+0000","last_undegraded":"2024-09-13T05:04:38.010912+0000","last_fullsized":"2024-09-13T05:04:38.010912+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'790","last_scrub_stamp":"2024-09-13T05:04:38.010887+0000","last_deep_scrub":"132'790","last_deep_scrub_stamp":"2024-09-13T05:04:38.010887+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.010887+0000","objects_scrubbed":14,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:35:41.131532+0000","scrub_duration":0.010720327,"objects_trimmed":0,"snaptrim_duration":0.001134092,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":3837,"num_read_kb":27013,"num_write":893,"num_write_kb":24244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":37,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:35.024024+0000","last_change":"2024-09-13T05:04:35.024136+0000","last_active":"2024-09-13T05:04:35.024024+0000","last_peered":"2024-09-13T05:04:35.024024+0000","last_clean":"2024-09-13T05:04:35.024024+0000","last_became_active":"2024-09-13T05:04:00.893437+0000","last_became_peered":"2024-09-13T05:04:00.893437+0000","last_unstale":"2024-09-13T05:04:35.024024+0000","last_undegraded":"2024-09-13T05:04:35.024024+0000","last_fullsized":"2024-09-13T05:04:35.024024+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:35.024003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:35.024003+0000","last_clean_scrub_stamp":"2024-09-13T05:04:35.024003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:41:50.798100+0000","scrub_duration":0.0017496670000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"132'722","reported_seq":3485,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:55.140444+0000","last_change":"2024-09-13T05:04:55.140597+0000","last_active":"2024-09-13T05:04:55.140444+0000","last_peered":"2024-09-13T05:04:55.140444+0000","last_clean":"2024-09-13T05:04:55.140444+0000","last_became_active":"2024-09-13T05:02:06.879710+0000","last_became_peered":"2024-09-13T05:02:06.879710+0000","last_unstale":"2024-09-13T05:04:55.140444+0000","last_undegraded":"2024-09-13T05:04:55.140444+0000","last_fullsized":"2024-09-13T05:04:55.140444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'722","last_scrub_stamp":"2024-09-13T05:04:55.140421+0000","last_deep_scrub":"132'722","last_deep_scrub_stamp":"2024-09-13T05:04:55.140421+0000","last_clean_scrub_stamp":"2024-09-13T05:04:55.140421+0000","objects_scrubbed":9,"log_size":722,"log_dups_size":0,"ondisk_log_size":722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T11:25:45.919847+0000","scrub_duration":0.0087987559999999996,"objects_trimmed":0,"snaptrim_duration":0.0012454269999999999,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4741,"num_read_kb":21460,"num_write":778,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"132'468","reported_seq":2610,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:28.128304+0000","last_change":"2024-09-13T05:04:28.128424+0000","last_active":"2024-09-13T05:04:28.128304+0000","last_peered":"2024-09-13T05:04:28.128304+0000","last_clean":"2024-09-13T05:04:28.128304+0000","last_became_active":"2024-09-13T05:02:06.877768+0000","last_became_peered":"2024-09-13T05:02:06.877768+0000","last_unstale":"2024-09-13T05:04:28.128304+0000","last_undegraded":"2024-09-13T05:04:28.128304+0000","last_fullsized":"2024-09-13T05:04:28.128304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'468","last_scrub_stamp":"2024-09-13T05:04:28.128285+0000","last_deep_scrub":"132'468","last_deep_scrub_stamp":"2024-09-13T05:04:28.128285+0000","last_clean_scrub_stamp":"2024-09-13T05:04:28.128285+0000","objects_scrubbed":9,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T05:34:23.536411+0000","scrub_duration":0.0092365989999999999,"objects_trimmed":0,"snaptrim_duration":0.0014054880000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2890,"num_read_kb":22965,"num_write":498,"num_write_kb":11867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"132'119","reported_seq":242,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:39.078342+0000","last_change":"2024-09-13T05:04:39.078450+0000","last_active":"2024-09-13T05:04:39.078342+0000","last_peered":"2024-09-13T05:04:39.078342+0000","last_clean":"2024-09-13T05:04:39.078342+0000","last_became_active":"2024-09-13T05:04:16.985644+0000","last_became_peered":"2024-09-13T05:04:16.985644+0000","last_unstale":"2024-09-13T05:04:39.078342+0000","last_undegraded":"2024-09-13T05:04:39.078342+0000","last_fullsized":"2024-09-13T05:04:39.078342+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'119","last_scrub_stamp":"2024-09-13T05:04:39.078318+0000","last_deep_scrub":"132'119","last_deep_scrub_stamp":"2024-09-13T05:04:39.078318+0000","last_clean_scrub_stamp":"2024-09-13T05:04:39.078318+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:25:29.602198+0000","scrub_duration":0.0042055560000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"132'2","reported_seq":146,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:27.564836+0000","last_change":"2024-09-13T05:04:27.564916+0000","last_active":"2024-09-13T05:04:27.564836+0000","last_peered":"2024-09-13T05:04:27.564836+0000","last_clean":"2024-09-13T05:04:27.564836+0000","last_became_active":"2024-09-13T05:01:57.175659+0000","last_became_peered":"2024-09-13T05:01:57.175659+0000","last_unstale":"2024-09-13T05:04:27.564836+0000","last_undegraded":"2024-09-13T05:04:27.564836+0000","last_fullsized":"2024-09-13T05:04:27.564836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'2","last_scrub_stamp":"2024-09-13T05:04:27.564817+0000","last_deep_scrub":"132'2","last_deep_scrub_stamp":"2024-09-13T05:04:27.564817+0000","last_clean_scrub_stamp":"2024-09-13T05:04:27.564817+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:46:25.023841+0000","scrub_duration":0.0032450019999999999,"objects_trimmed":0,"snaptrim_duration":0.00048067800000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"132'627","reported_seq":4256,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:40.520316+0000","last_change":"2024-09-13T05:04:40.520443+0000","last_active":"2024-09-13T05:04:40.520316+0000","last_peered":"2024-09-13T05:04:40.520316+0000","last_clean":"2024-09-13T05:04:40.520316+0000","last_became_active":"2024-09-13T05:02:06.882386+0000","last_became_peered":"2024-09-13T05:02:06.882386+0000","last_unstale":"2024-09-13T05:04:40.520316+0000","last_undegraded":"2024-09-13T05:04:40.520316+0000","last_fullsized":"2024-09-13T05:04:40.520316+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'627","last_scrub_stamp":"2024-09-13T05:04:40.520292+0000","last_deep_scrub":"132'627","last_deep_scrub_stamp":"2024-09-13T05:04:40.520292+0000","last_clean_scrub_stamp":"2024-09-13T05:04:40.520292+0000","objects_scrubbed":19,"log_size":627,"log_dups_size":0,"ondisk_log_size":627,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:51:35.622564+0000","scrub_duration":0.016126941999999998,"objects_trimmed":0,"snaptrim_duration":0.000801774,"stat_sum":{"num_bytes":0,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":19,"num_read":5956,"num_read_kb":38981,"num_write":598,"num_write_kb":13183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"132'3","reported_seq":98,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:52.783228+0000","last_change":"2024-09-13T05:04:52.783328+0000","last_active":"2024-09-13T05:04:52.783228+0000","last_peered":"2024-09-13T05:04:52.783228+0000","last_clean":"2024-09-13T05:04:52.783228+0000","last_became_active":"2024-09-13T05:04:16.988660+0000","last_became_peered":"2024-09-13T05:04:16.988660+0000","last_unstale":"2024-09-13T05:04:52.783228+0000","last_undegraded":"2024-09-13T05:04:52.783228+0000","last_fullsized":"2024-09-13T05:04:52.783228+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'3","last_scrub_stamp":"2024-09-13T05:04:52.783205+0000","last_deep_scrub":"132'3","last_deep_scrub_stamp":"2024-09-13T05:04:52.783205+0000","last_clean_scrub_stamp":"2024-09-13T05:04:52.783205+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:16:09.459447+0000","scrub_duration":0.0032525829999999999,"objects_trimmed":0,"snaptrim_duration":0.00031038799999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"132'621","reported_seq":3722,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:51.828227+0000","last_change":"2024-09-13T05:04:51.828381+0000","last_active":"2024-09-13T05:04:51.828227+0000","last_peered":"2024-09-13T05:04:51.828227+0000","last_clean":"2024-09-13T05:04:51.828227+0000","last_became_active":"2024-09-13T05:04:16.989227+0000","last_became_peered":"2024-09-13T05:04:16.989227+0000","last_unstale":"2024-09-13T05:04:51.828227+0000","last_undegraded":"2024-09-13T05:04:51.828227+0000","last_fullsized":"2024-09-13T05:04:51.828227+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'621","last_scrub_stamp":"2024-09-13T05:04:51.828204+0000","last_deep_scrub":"132'621","last_deep_scrub_stamp":"2024-09-13T05:04:51.828204+0000","last_clean_scrub_stamp":"2024-09-13T05:04:51.828204+0000","objects_scrubbed":10,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T16:26:40.897660+0000","scrub_duration":0.0089663810000000007,"objects_trimmed":0,"snaptrim_duration":0.001801534,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":3892,"num_read_kb":35375,"num_write":569,"num_write_kb":11902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":14282,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":143,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:30.333640+0000","last_change":"2024-09-13T05:04:30.333750+0000","last_active":"2024-09-13T05:04:30.333640+0000","last_peered":"2024-09-13T05:04:30.333640+0000","last_clean":"2024-09-13T05:04:30.333640+0000","last_became_active":"2024-09-13T05:01:57.177619+0000","last_became_peered":"2024-09-13T05:01:57.177619+0000","last_unstale":"2024-09-13T05:04:30.333640+0000","last_undegraded":"2024-09-13T05:04:30.333640+0000","last_fullsized":"2024-09-13T05:04:30.333640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:30.333616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:30.333616+0000","last_clean_scrub_stamp":"2024-09-13T05:04:30.333616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:13:23.974750+0000","scrub_duration":0.001719145,"objects_trimmed":0,"snaptrim_duration":0.00031820200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"132'1194","reported_seq":3939,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:36.002937+0000","last_change":"2024-09-13T05:04:36.003059+0000","last_active":"2024-09-13T05:04:36.002937+0000","last_peered":"2024-09-13T05:04:36.002937+0000","last_clean":"2024-09-13T05:04:36.002937+0000","last_became_active":"2024-09-13T05:04:04.921357+0000","last_became_peered":"2024-09-13T05:04:04.921357+0000","last_unstale":"2024-09-13T05:04:36.002937+0000","last_undegraded":"2024-09-13T05:04:36.002937+0000","last_fullsized":"2024-09-13T05:04:36.002937+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'1194","last_scrub_stamp":"2024-09-13T05:04:36.002913+0000","last_deep_scrub":"132'1194","last_deep_scrub_stamp":"2024-09-13T05:04:36.002913+0000","last_clean_scrub_stamp":"2024-09-13T05:04:36.002913+0000","objects_scrubbed":14,"log_size":1194,"log_dups_size":0,"ondisk_log_size":1194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:50:40.733133+0000","scrub_duration":0.011374762,"objects_trimmed":0,"snaptrim_duration":0.00056158599999999999,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":7514,"num_read_kb":16397,"num_write":1820,"num_write_kb":6236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":95,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:32.923809+0000","last_change":"2024-09-13T05:04:32.923909+0000","last_active":"2024-09-13T05:04:32.923809+0000","last_peered":"2024-09-13T05:04:32.923809+0000","last_clean":"2024-09-13T05:04:32.923809+0000","last_became_active":"2024-09-13T05:04:16.988793+0000","last_became_peered":"2024-09-13T05:04:16.988793+0000","last_unstale":"2024-09-13T05:04:32.923809+0000","last_undegraded":"2024-09-13T05:04:32.923809+0000","last_fullsized":"2024-09-13T05:04:32.923809+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-13T05:04:32.923787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-13T05:04:32.923787+0000","last_clean_scrub_stamp":"2024-09-13T05:04:32.923787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T06:56:57.913618+0000","scrub_duration":0.001859943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"132'454","reported_seq":3753,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:40.059449+0000","last_change":"2024-09-13T05:04:40.059573+0000","last_active":"2024-09-13T05:04:40.059449+0000","last_peered":"2024-09-13T05:04:40.059449+0000","last_clean":"2024-09-13T05:04:40.059449+0000","last_became_active":"2024-09-13T05:04:04.922863+0000","last_became_peered":"2024-09-13T05:04:04.922863+0000","last_unstale":"2024-09-13T05:04:40.059449+0000","last_undegraded":"2024-09-13T05:04:40.059449+0000","last_fullsized":"2024-09-13T05:04:40.059449+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'454","last_scrub_stamp":"2024-09-13T05:04:40.059426+0000","last_deep_scrub":"132'454","last_deep_scrub_stamp":"2024-09-13T05:04:40.059426+0000","last_clean_scrub_stamp":"2024-09-13T05:04:40.059426+0000","objects_scrubbed":8,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:59:39.040897+0000","scrub_duration":0.0073298180000000001,"objects_trimmed":0,"snaptrim_duration":0.000704993,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":3047,"num_read_kb":25131,"num_write":473,"num_write_kb":10911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"132'470","reported_seq":2894,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:37.339240+0000","last_change":"2024-09-13T05:04:37.339364+0000","last_active":"2024-09-13T05:04:37.339240+0000","last_peered":"2024-09-13T05:04:37.339240+0000","last_clean":"2024-09-13T05:04:37.339240+0000","last_became_active":"2024-09-13T05:02:06.876472+0000","last_became_peered":"2024-09-13T05:02:06.876472+0000","last_unstale":"2024-09-13T05:04:37.339240+0000","last_undegraded":"2024-09-13T05:04:37.339240+0000","last_fullsized":"2024-09-13T05:04:37.339240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'470","last_scrub_stamp":"2024-09-13T05:04:37.339217+0000","last_deep_scrub":"132'470","last_deep_scrub_stamp":"2024-09-13T05:04:37.339217+0000","last_clean_scrub_stamp":"2024-09-13T05:04:37.339217+0000","objects_scrubbed":7,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:54:44.880836+0000","scrub_duration":0.0069644119999999997,"objects_trimmed":0,"snaptrim_duration":0.00088059,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":2781,"num_read_kb":23096,"num_write":524,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"132'660","reported_seq":4688,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:32.942455+0000","last_change":"2024-09-13T05:04:32.942588+0000","last_active":"2024-09-13T05:04:32.942455+0000","last_peered":"2024-09-13T05:04:32.942455+0000","last_clean":"2024-09-13T05:04:32.942455+0000","last_became_active":"2024-09-13T05:04:14.971688+0000","last_became_peered":"2024-09-13T05:04:14.971688+0000","last_unstale":"2024-09-13T05:04:32.942455+0000","last_undegraded":"2024-09-13T05:04:32.942455+0000","last_fullsized":"2024-09-13T05:04:32.942455+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'660","last_scrub_stamp":"2024-09-13T05:04:32.942433+0000","last_deep_scrub":"132'660","last_deep_scrub_stamp":"2024-09-13T05:04:32.942433+0000","last_clean_scrub_stamp":"2024-09-13T05:04:32.942433+0000","objects_scrubbed":17,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T13:21:17.553587+0000","scrub_duration":0.013735855999999999,"objects_trimmed":0,"snaptrim_duration":0.000830867,"stat_sum":{"num_bytes":0,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":16,"num_read":5002,"num_read_kb":15952,"num_write":672,"num_write_kb":24389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"132'894","reported_seq":2849,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:30.987588+0000","last_change":"2024-09-13T05:04:30.987717+0000","last_active":"2024-09-13T05:04:30.987588+0000","last_peered":"2024-09-13T05:04:30.987588+0000","last_clean":"2024-09-13T05:04:30.987588+0000","last_became_active":"2024-09-13T05:04:04.921071+0000","last_became_peered":"2024-09-13T05:04:04.921071+0000","last_unstale":"2024-09-13T05:04:30.987588+0000","last_undegraded":"2024-09-13T05:04:30.987588+0000","last_fullsized":"2024-09-13T05:04:30.987588+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'894","last_scrub_stamp":"2024-09-13T05:04:30.987564+0000","last_deep_scrub":"132'894","last_deep_scrub_stamp":"2024-09-13T05:04:30.987564+0000","last_clean_scrub_stamp":"2024-09-13T05:04:30.987564+0000","objects_scrubbed":9,"log_size":894,"log_dups_size":0,"ondisk_log_size":894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T14:53:45.863587+0000","scrub_duration":0.0086435769999999995,"objects_trimmed":0,"snaptrim_duration":0.000918869,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4685,"num_read_kb":9580,"num_write":1234,"num_write_kb":3447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"132'449","reported_seq":3826,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:45.090968+0000","last_change":"2024-09-13T05:04:45.091086+0000","last_active":"2024-09-13T05:04:45.090968+0000","last_peered":"2024-09-13T05:04:45.090968+0000","last_clean":"2024-09-13T05:04:45.090968+0000","last_became_active":"2024-09-13T05:04:03.919341+0000","last_became_peered":"2024-09-13T05:04:03.919341+0000","last_unstale":"2024-09-13T05:04:45.090968+0000","last_undegraded":"2024-09-13T05:04:45.090968+0000","last_fullsized":"2024-09-13T05:04:45.090968+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'449","last_scrub_stamp":"2024-09-13T05:04:45.090930+0000","last_deep_scrub":"132'449","last_deep_scrub_stamp":"2024-09-13T05:04:45.090930+0000","last_clean_scrub_stamp":"2024-09-13T05:04:45.090930+0000","objects_scrubbed":12,"log_size":449,"log_dups_size":0,"ondisk_log_size":449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:49:37.485192+0000","scrub_duration":0.0089008170000000001,"objects_trimmed":0,"snaptrim_duration":0.00115329,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3711,"num_read_kb":10580,"num_write":371,"num_write_kb":2870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"132'581","reported_seq":5315,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:05:04.826339+0000","last_change":"2024-09-13T05:05:04.826460+0000","last_active":"2024-09-13T05:05:04.826339+0000","last_peered":"2024-09-13T05:05:04.826339+0000","last_clean":"2024-09-13T05:05:04.826339+0000","last_became_active":"2024-09-13T05:04:16.984183+0000","last_became_peered":"2024-09-13T05:04:16.984183+0000","last_unstale":"2024-09-13T05:05:04.826339+0000","last_undegraded":"2024-09-13T05:05:04.826339+0000","last_fullsized":"2024-09-13T05:05:04.826339+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'581","last_scrub_stamp":"2024-09-13T05:05:04.826316+0000","last_deep_scrub":"132'581","last_deep_scrub_stamp":"2024-09-13T05:05:04.826316+0000","last_clean_scrub_stamp":"2024-09-13T05:05:04.826316+0000","objects_scrubbed":11,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T08:07:18.614149+0000","scrub_duration":0.0097281160000000002,"objects_trimmed":0,"snaptrim_duration":0.00064799200000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":7557,"num_read_kb":36552,"num_write":609,"num_write_kb":7912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"132'589","reported_seq":5637,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:45.966973+0000","last_change":"2024-09-13T05:04:45.967134+0000","last_active":"2024-09-13T05:04:45.966973+0000","last_peered":"2024-09-13T05:04:45.966973+0000","last_clean":"2024-09-13T05:04:45.966973+0000","last_became_active":"2024-09-13T05:04:16.982045+0000","last_became_peered":"2024-09-13T05:04:16.982045+0000","last_unstale":"2024-09-13T05:04:45.966973+0000","last_undegraded":"2024-09-13T05:04:45.966973+0000","last_fullsized":"2024-09-13T05:04:45.966973+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'589","last_scrub_stamp":"2024-09-13T05:04:45.966929+0000","last_deep_scrub":"132'589","last_deep_scrub_stamp":"2024-09-13T05:04:45.966929+0000","last_clean_scrub_stamp":"2024-09-13T05:04:45.966929+0000","objects_scrubbed":14,"log_size":589,"log_dups_size":0,"ondisk_log_size":589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T12:33:04.494166+0000","scrub_duration":0.011784565,"objects_trimmed":0,"snaptrim_duration":0.001156331,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":6411,"num_read_kb":60505,"num_write":679,"num_write_kb":30484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"132'399","reported_seq":3057,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:38.057065+0000","last_change":"2024-09-13T05:04:38.057170+0000","last_active":"2024-09-13T05:04:38.057065+0000","last_peered":"2024-09-13T05:04:38.057065+0000","last_clean":"2024-09-13T05:04:38.057065+0000","last_became_active":"2024-09-13T05:04:16.982033+0000","last_became_peered":"2024-09-13T05:04:16.982033+0000","last_unstale":"2024-09-13T05:04:38.057065+0000","last_undegraded":"2024-09-13T05:04:38.057065+0000","last_fullsized":"2024-09-13T05:04:38.057065+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'399","last_scrub_stamp":"2024-09-13T05:04:38.057039+0000","last_deep_scrub":"132'399","last_deep_scrub_stamp":"2024-09-13T05:04:38.057039+0000","last_clean_scrub_stamp":"2024-09-13T05:04:38.057039+0000","objects_scrubbed":9,"log_size":399,"log_dups_size":0,"ondisk_log_size":399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T17:03:45.966737+0000","scrub_duration":0.0072941580000000002,"objects_trimmed":0,"snaptrim_duration":0.0018351140000000001,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":3115,"num_read_kb":14090,"num_write":341,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":10232732,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"132'886","reported_seq":3127,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:31.930985+0000","last_change":"2024-09-13T05:04:31.931128+0000","last_active":"2024-09-13T05:04:31.930985+0000","last_peered":"2024-09-13T05:04:31.930985+0000","last_clean":"2024-09-13T05:04:31.930985+0000","last_became_active":"2024-09-13T05:04:14.970105+0000","last_became_peered":"2024-09-13T05:04:14.970105+0000","last_unstale":"2024-09-13T05:04:31.930985+0000","last_undegraded":"2024-09-13T05:04:31.930985+0000","last_fullsized":"2024-09-13T05:04:31.930985+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'886","last_scrub_stamp":"2024-09-13T05:04:31.930960+0000","last_deep_scrub":"132'886","last_deep_scrub_stamp":"2024-09-13T05:04:31.930960+0000","last_clean_scrub_stamp":"2024-09-13T05:04:31.930960+0000","objects_scrubbed":12,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:12:22.292092+0000","scrub_duration":0.010054027,"objects_trimmed":0,"snaptrim_duration":0.0013036269999999999,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":5378,"num_read_kb":11761,"num_write":1169,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"132'559","reported_seq":4262,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:34.957996+0000","last_change":"2024-09-13T05:04:34.958114+0000","last_active":"2024-09-13T05:04:34.957996+0000","last_peered":"2024-09-13T05:04:34.957996+0000","last_clean":"2024-09-13T05:04:34.957996+0000","last_became_active":"2024-09-13T05:04:16.981164+0000","last_became_peered":"2024-09-13T05:04:16.981164+0000","last_unstale":"2024-09-13T05:04:34.957996+0000","last_undegraded":"2024-09-13T05:04:34.957996+0000","last_fullsized":"2024-09-13T05:04:34.957996+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'559","last_scrub_stamp":"2024-09-13T05:04:34.957972+0000","last_deep_scrub":"132'559","last_deep_scrub_stamp":"2024-09-13T05:04:34.957972+0000","last_clean_scrub_stamp":"2024-09-13T05:04:34.957972+0000","objects_scrubbed":8,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T09:14:42.577985+0000","scrub_duration":0.0076218420000000002,"objects_trimmed":0,"snaptrim_duration":0.0012558109999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":5196,"num_read_kb":13539,"num_write":508,"num_write_kb":2572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"132'476","reported_seq":3869,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:52.181796+0000","last_change":"2024-09-13T05:04:52.181958+0000","last_active":"2024-09-13T05:04:52.181796+0000","last_peered":"2024-09-13T05:04:52.181796+0000","last_clean":"2024-09-13T05:04:52.181796+0000","last_became_active":"2024-09-13T05:02:06.883749+0000","last_became_peered":"2024-09-13T05:02:06.883749+0000","last_unstale":"2024-09-13T05:04:52.181796+0000","last_undegraded":"2024-09-13T05:04:52.181796+0000","last_fullsized":"2024-09-13T05:04:52.181796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'476","last_scrub_stamp":"2024-09-13T05:04:52.181760+0000","last_deep_scrub":"132'476","last_deep_scrub_stamp":"2024-09-13T05:04:52.181760+0000","last_clean_scrub_stamp":"2024-09-13T05:04:52.181760+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T07:16:26.397182+0000","scrub_duration":0.010217489999999999,"objects_trimmed":0,"snaptrim_duration":0.00095456399999999998,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":4307,"num_read_kb":36017,"num_write":458,"num_write_kb":18031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"132'771","reported_seq":3303,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-13T05:04:29.965841+0000","last_change":"2024-09-13T05:04:29.965960+0000","last_active":"2024-09-13T05:04:29.965841+0000","last_peered":"2024-09-13T05:04:29.965841+0000","last_clean":"2024-09-13T05:04:29.965841+0000","last_became_active":"2024-09-13T05:02:06.887497+0000","last_became_peered":"2024-09-13T05:02:06.887497+0000","last_unstale":"2024-09-13T05:04:29.965841+0000","last_undegraded":"2024-09-13T05:04:29.965841+0000","last_fullsized":"2024-09-13T05:04:29.965841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"132'771","last_scrub_stamp":"2024-09-13T05:04:29.965819+0000","last_deep_scrub":"132'771","last_deep_scrub_stamp":"2024-09-13T05:04:29.965819+0000","last_clean_scrub_stamp":"2024-09-13T05:04:29.965819+0000","objects_scrubbed":10,"log_size":771,"log_dups_size":0,"ondisk_log_size":771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-14T15:25:52.349432+0000","scrub_duration":0.0094627819999999994,"objects_trimmed":0,"snaptrim_duration":0.00039001800000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":4562,"num_read_kb":20898,"num_write":1077,"num_write_kb":9944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":339,"num_object_clones":0,"num_object_copies":678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":339,"num_whiteouts":335,"num_read":148133,"num_read_kb":707003,"num_write":23736,"num_write_kb":381686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2650,"num_bytes_recovered":10247014,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":20485,"ondisk_log_size":20485,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9702,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58956,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312884,"statfs":{"total":96636764160,"available":96576393216,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":58718667},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.30199999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.72399999999999998,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.45400000000000001,"15min":0.44800000000000001},"min":{"1min":0.34300000000000003,"5min":0.22,"15min":0.22},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Sep 13 05:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46800000000000003,"15min":0.46400000000000002},"min":{"1min":0.41699999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.373,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71799999999999997,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.44600000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.48599999999999999,"15min":0.47899999999999998},"min":{"1min":0.41299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73399999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.497},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.497,"15min":0.50600000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.748,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Sep 13 05:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.32200000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.38100000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73099999999999998,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.433}]},{"osd":4,"last update":"Fri Sep 13 05:05:02 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.39800000000000002,"15min":0.38800000000000001},"min":{"1min":0.35299999999999998,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.374},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.45500000000000002,"15min":0.45100000000000001},"min":{"1min":0.38,"5min":0.253,"15min":0.253},"max":{"1min":0.68300000000000005,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.41299999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.40600000000000003,"15min":0.39700000000000002},"min":{"1min":0.33000000000000002,"5min":0.221,"15min":0.221},"max":{"1min":0.70299999999999996,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.438,"15min":0.42499999999999999},"min":{"1min":0.40999999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.53700000000000003}]},{"osd":6,"last update":"Fri Sep 13 05:04:58 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.45900000000000002,"15min":0.45500000000000002},"min":{"1min":0.372,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.67700000000000005,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.48099999999999998,"15min":0.47099999999999997},"min":{"1min":0.43099999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.56200000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40848,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94330992,"statfs":{"total":96636764160,"available":96594935808,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40237511},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.42899999999999999,"15min":0.40300000000000002},"min":{"1min":0.45400000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.48099999999999998,"15min":0.46899999999999997},"min":{"1min":0.46500000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59199999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.44500000000000001,"15min":0.41899999999999998},"min":{"1min":0.38300000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.46899999999999997,"15min":0.45300000000000001},"min":{"1min":0.441,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73899999999999999}]},{"osd":2,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.46500000000000002,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.47399999999999998,"15min":0.45600000000000002},"min":{"1min":0.41399999999999998,"5min":0.313,"15min":0.313},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46400000000000002,"15min":0.44800000000000001},"min":{"1min":0.375,"5min":0.249,"15min":0.249},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.48899999999999999,"15min":0.48299999999999998},"min":{"1min":0.436,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.76700000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.314,"5min":0.224,"15min":0.224},"max":{"1min":0.63300000000000001,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.42599999999999999,"15min":0.40999999999999998},"min":{"1min":0.33800000000000002,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.71399999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.47199999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.40999999999999998,"15min":0.39600000000000002},"min":{"1min":0.35899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.45400000000000001,"15min":0.44},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71399999999999997,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.432}]},{"osd":7,"last update":"Fri Sep 13 05:05:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.45100000000000001,"15min":0.433},"min":{"1min":0.35299999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.68200000000000005,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.46800000000000003,"15min":0.45500000000000002},"min":{"1min":0.38200000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.69699999999999995,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.50800000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54800,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94317040,"statfs":{"total":96636764160,"available":96580648960,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54524358},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.442,"15min":0.42099999999999999},"min":{"1min":0.26400000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.502},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.45500000000000002,"15min":0.42699999999999999},"min":{"1min":0.36499999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.63200000000000001}]},{"osd":1,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.47499999999999998,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.47199999999999998,"15min":0.45300000000000001},"min":{"1min":0.39900000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72399999999999998,"5min":0.875,"15min":0.875},"last":0.51500000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.45700000000000002,"15min":0.436},"min":{"1min":0.32400000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.49299999999999999,"15min":0.47599999999999998},"min":{"1min":0.35699999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72599999999999998,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.47799999999999998,"15min":0.45400000000000001},"min":{"1min":0.38100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.48299999999999998,"15min":0.46200000000000002},"min":{"1min":0.33900000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.40600000000000003,"15min":0.40500000000000003},"min":{"1min":0.28799999999999998,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65100000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.42299999999999999},"min":{"1min":0.314,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69299999999999995,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.33400000000000002}]},{"osd":6,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.44600000000000001,"15min":0.45100000000000001},"min":{"1min":0.35299999999999998,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59599999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45600000000000002},"min":{"1min":0.35899999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69199999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.38500000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:04:59 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47899999999999998,"15min":0.48499999999999999},"min":{"1min":0.34899999999999998,"5min":0.315,"15min":0.315},"max":{"1min":0.67000000000000004,"5min":1.006,"15min":1.006},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.50700000000000001,"15min":0.505},"min":{"1min":0.35699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.68100000000000005,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.41699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72852,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298988,"statfs":{"total":96636764160,"available":96562163712,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69699999999999995,"15min":0.753},"min":{"1min":0.372,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":3.2400000000000002,"5min":6.5469999999999997,"15min":6.5469999999999997},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68600000000000005,"15min":0.72899999999999998},"min":{"1min":0.47099999999999997,"5min":0.216,"15min":0.216},"max":{"1min":2.681,"5min":6.4509999999999996,"15min":6.4509999999999996},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.432,"5min":0.252,"15min":0.252},"max":{"1min":3.3730000000000002,"5min":3.3730000000000002,"15min":3.3730000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.68100000000000005,"15min":0.74299999999999999},"min":{"1min":0.438,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.0700000000000001,"5min":6.5190000000000001,"15min":6.5190000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.52800000000000002,"15min":0.47999999999999998},"min":{"1min":0.47599999999999998,"5min":0.247,"15min":0.247},"max":{"1min":3.4380000000000002,"5min":3.4380000000000002,"15min":3.4380000000000002},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71499999999999997,"15min":0.78300000000000003},"min":{"1min":0.41899999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.3129999999999999,"5min":6.4880000000000004,"15min":6.4880000000000004},"last":0.61799999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.48199999999999998,"15min":0.46400000000000002},"min":{"1min":0.44600000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.64700000000000002,"15min":0.70299999999999996},"min":{"1min":0.378,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68899999999999995,"5min":3.3460000000000001,"15min":3.3460000000000001},"last":0.66500000000000004}]},{"osd":5,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.60399999999999998,"15min":0.68000000000000005},"min":{"1min":0.33600000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.746,"5min":6.577,"15min":6.577},"last":0.433},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.51100000000000001,"15min":0.54300000000000004},"min":{"1min":0.33300000000000002,"5min":0.214,"15min":0.214},"max":{"1min":0.59199999999999997,"5min":3.379,"15min":3.379},"last":0.58699999999999997}]},{"osd":6,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.32000000000000001,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52100000000000002,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.64600000000000002,"5min":2.371,"15min":2.371},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.66500000000000004,"15min":0.73699999999999999},"min":{"1min":0.36299999999999999,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71699999999999997,"5min":6.6020000000000003,"15min":6.6020000000000003},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.59099999999999997},"min":{"1min":0.35699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.67400000000000004,"5min":3.6869999999999998,"15min":3.6869999999999998},"last":0.60699999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574890,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68752,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94303088,"statfs":{"total":96636764160,"available":96566362112,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":68811208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.34000000000000002,"15min":0.33000000000000002},"min":{"1min":0.19600000000000001,"5min":0.19,"15min":0.19},"max":{"1min":0.502,"5min":1.889,"15min":1.889},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38100000000000001,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.50900000000000001,"5min":1.204,"15min":1.204},"last":0.42399999999999999}]},{"osd":1,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.251,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.57399999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.318},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.40799999999999997,"15min":0.38800000000000001},"min":{"1min":0.25800000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.629,"5min":1.925,"15min":1.925},"last":0.37}]},{"osd":2,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.40200000000000002,"15min":0.40100000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.58599999999999997,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.42799999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.68600000000000005,"5min":1.8080000000000001,"15min":1.8080000000000001},"last":0.39800000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.54300000000000004,"15min":0.52400000000000002},"min":{"1min":0.41699999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.006,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.55600000000000005,"15min":0.55100000000000005},"min":{"1min":0.46899999999999997,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.94699999999999995,"5min":1.159,"15min":1.159},"last":0.73799999999999999}]},{"osd":5,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.84899999999999998,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.006,"5min":2.8090000000000002,"15min":2.8090000000000002},"last":0.79200000000000004}]},{"osd":6,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.58199999999999996,"15min":0.55900000000000005},"min":{"1min":0.42299999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.94899999999999995,"5min":1.772,"15min":1.772},"last":0.753},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.38400000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.81899999999999995,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.67900000000000005}]},{"osd":7,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59399999999999997,"15min":0.59099999999999997},"min":{"1min":0.443,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.93000000000000005,"5min":1.222,"15min":1.222},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.65700000000000003,"15min":0.67700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.97299999999999998,"5min":3.02,"15min":3.02},"last":0.77300000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28560,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343280,"statfs":{"total":96636764160,"available":96607518720,"internally_reserved":0,"allocated":1581056,"data_stored":757274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35999999999999999,"15min":0.36599999999999999},"min":{"1min":0.24099999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53700000000000003,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.36699999999999999,"15min":0.375},"min":{"1min":0.25,"5min":0.218,"15min":0.218},"max":{"1min":0.59499999999999997,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.371}]},{"osd":1,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.36599999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.54200000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.38100000000000001,"15min":0.38600000000000001},"min":{"1min":0.28199999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56200000000000006,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.39400000000000002,"15min":0.40200000000000002},"min":{"1min":0.313,"5min":0.249,"15min":0.249},"max":{"1min":0.51700000000000002,"5min":1,"15min":1},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42499999999999999,"15min":0.439},"min":{"1min":0.30299999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.57599999999999996,"5min":1.109,"15min":1.109},"last":0.33500000000000002}]},{"osd":4,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.54500000000000004,"15min":0.54800000000000004},"min":{"1min":0.43099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.81000000000000005,"5min":1.046,"15min":1.046},"last":0.495},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.44500000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.68400000000000005,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54800000000000004}]},{"osd":5,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.38500000000000001,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71499999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.54200000000000004,"15min":0.53100000000000003},"min":{"1min":0.439,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.53800000000000003,"15min":0.52900000000000003},"min":{"1min":0.46899999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.39100000000000001,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.82499999999999996,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.56799999999999995}]},{"osd":7,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.79300000000000004,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.56999999999999995,"15min":0.56899999999999995},"min":{"1min":0.47799999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72908,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":71294,"kb_avail":94298932,"statfs":{"total":96636764160,"available":96562106368,"internally_reserved":0,"allocated":1642496,"data_stored":1347661,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":73005510},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.33300000000000002,"15min":0.32300000000000001},"min":{"1min":0.20399999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.63,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.371,"15min":0.35099999999999998},"min":{"1min":0.23999999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.34200000000000003,"15min":0.32800000000000001},"min":{"1min":0.25900000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40100000000000002},"min":{"1min":0.27500000000000002,"5min":0.222,"15min":0.222},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.45700000000000002,"15min":0.441},"min":{"1min":0.34100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.41999999999999998,"15min":0.40000000000000002},"min":{"1min":0.29299999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.441}]},{"osd":4,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.51700000000000002,"15min":0.501},"min":{"1min":0.38100000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.53100000000000003,"15min":0.51600000000000001},"min":{"1min":0.46899999999999997,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84999999999999998,"5min":0.87,"15min":0.87},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.55400000000000005,"15min":0.54400000000000004},"min":{"1min":0.435,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.52500000000000002},"min":{"1min":0.42199999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.55100000000000005,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83399999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Sep 13 05:05:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.49399999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.629},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.59499999999999997,"15min":0.58099999999999996},"min":{"1min":0.45500000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.76500000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58900,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312940,"statfs":{"total":96636764160,"available":96576450560,"internally_reserved":0,"allocated":1585152,"data_stored":757293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58718663},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.47399999999999998,"15min":0.374},"min":{"1min":0.22500000000000001,"5min":0.184,"15min":0.184},"max":{"1min":0.71099999999999997,"5min":11.625,"15min":11.625},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.42599999999999999,"15min":0.38400000000000001},"min":{"1min":0.32500000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.65100000000000002,"5min":4.9660000000000002,"15min":4.9660000000000002},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.438,"15min":0.378},"min":{"1min":0.26800000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.73999999999999999,"5min":7.2080000000000002,"15min":7.2080000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.375,"15min":0.36399999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.56499999999999995,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.38600000000000001}]},{"osd":3,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.39300000000000002,"15min":0.38200000000000001},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.60299999999999998,"5min":1.833,"15min":1.833},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.439,"5min":0.504,"15min":0.44600000000000001},"min":{"1min":0.29799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.76100000000000001,"5min":8.0340000000000007,"15min":8.0340000000000007},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.52800000000000002,"15min":0.502},"min":{"1min":0.41999999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.94399999999999995,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.58899999999999997,"15min":0.51900000000000002},"min":{"1min":0.42899999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":7.2939999999999996,"15min":7.2939999999999996},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.57699999999999996,"15min":0.53900000000000003},"min":{"1min":0.39500000000000002,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75800000000000001,"5min":3.7229999999999999,"15min":3.7229999999999999},"last":0.629},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.58699999999999997,"15min":0.55000000000000004},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88200000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.80800000000000005}]},{"osd":6,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.58699999999999997},"min":{"1min":0.44800000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.95199999999999996,"5min":9.1579999999999995,"15min":9.1579999999999995},"last":0.745},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68799999999999994,"15min":0.60799999999999998},"min":{"1min":0.495,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.995,"5min":9.1929999999999996,"15min":9.1929999999999996},"last":0.79800000000000004}]},{"osd":7,"last update":"Fri Sep 13 05:05:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.55800000000000005,"15min":0.54500000000000004},"min":{"1min":0.41699999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.80900000000000005,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.621,"15min":0.56699999999999995},"min":{"1min":0.499,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.91700000000000004,"5min":6.3730000000000002,"15min":6.3730000000000002},"last":0.77300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4851,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-13T05:05:10.835 DEBUG:teuthology.orchestra.run.smithi053:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-13T05:05:11.135 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-13T05:05:11.136 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-13T05:05:11.136 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-13T05:05:11.136 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.183 INFO:tasks.ceph.osd.0:Stopped 2024-09-13T05:05:11.183 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-13T05:05:11.183 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.220 INFO:tasks.ceph.osd.1:Stopped 2024-09-13T05:05:11.221 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-13T05:05:11.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.289 INFO:tasks.ceph.osd.2:Stopped 2024-09-13T05:05:11.289 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-13T05:05:11.289 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.360 INFO:tasks.ceph.osd.3:Stopped 2024-09-13T05:05:11.360 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-13T05:05:11.360 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.395 INFO:tasks.ceph.osd.4:Stopped 2024-09-13T05:05:11.395 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-13T05:05:11.396 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.466 INFO:tasks.ceph.osd.5:Stopped 2024-09-13T05:05:11.467 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-13T05:05:11.467 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.541 INFO:tasks.ceph.osd.6:Stopped 2024-09-13T05:05:11.542 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-13T05:05:11.542 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.614 INFO:tasks.ceph.osd.7:Stopped 2024-09-13T05:05:11.615 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-13T05:05:11.615 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-13T05:05:11.615 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.639 INFO:tasks.ceph.mgr.y:Stopped 2024-09-13T05:05:11.640 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-13T05:05:11.640 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.684 INFO:tasks.ceph.mgr.x:Stopped 2024-09-13T05:05:11.684 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-13T05:05:11.684 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-13T05:05:11.684 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.696 INFO:tasks.ceph.mon.a:Stopped 2024-09-13T05:05:11.696 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-13T05:05:11.696 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.749 INFO:tasks.ceph.mon.c:Stopped 2024-09-13T05:05:11.750 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-13T05:05:11.750 INFO:teuthology.orchestra.run:waiting for 300 2024-09-13T05:05:11.762 INFO:tasks.ceph.mon.b:Stopped 2024-09-13T05:05:11.762 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-13T05:05:11.762 DEBUG:teuthology.orchestra.run.smithi053:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-13T05:05:11.829 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:05:11.829 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-13T05:05:12.060 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:05:12.060 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-13T05:05:12.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:05:12.134 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-13T05:05:12.191 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:05:12.191 DEBUG:teuthology.orchestra.run.smithi053:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-13T05:05:12.261 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:05:12.261 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-13T05:05:12.413 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:05:12.413 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-13T05:05:12.483 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:05:12.483 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-13T05:05:12.539 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:05:12.539 DEBUG:teuthology.orchestra.run.smithi110:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-13T05:05:12.583 INFO:tasks.ceph:Archiving mon data... 2024-09-13T05:05:12.584 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/data/mon.a.tgz 2024-09-13T05:05:12.584 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-13T05:05:12.601 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.g5yMEFydrm 2024-09-13T05:05:12.601 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.g5yMEFydrm 2024-09-13T05:05:12.746 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.g5yMEFydrm 2024-09-13T05:05:12.830 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.g5yMEFydrm is 342KB 2024-09-13T05:05:12.926 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.g5yMEFydrm 2024-09-13T05:05:12.942 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/data/mon.c.tgz 2024-09-13T05:05:12.942 DEBUG:teuthology.orchestra.run.smithi053:> mktemp 2024-09-13T05:05:12.997 INFO:teuthology.orchestra.run.smithi053.stdout:/tmp/tmp.DJWzjgmfER 2024-09-13T05:05:12.998 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.DJWzjgmfER 2024-09-13T05:05:13.159 DEBUG:teuthology.orchestra.run.smithi053:> sudo chmod 0666 /tmp/tmp.DJWzjgmfER 2024-09-13T05:05:13.243 DEBUG:teuthology.orchestra.remote:smithi053:/tmp/tmp.DJWzjgmfER is 471KB 2024-09-13T05:05:13.341 DEBUG:teuthology.orchestra.run.smithi053:> rm -fr /tmp/tmp.DJWzjgmfER 2024-09-13T05:05:13.356 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/data/mon.b.tgz 2024-09-13T05:05:13.357 DEBUG:teuthology.orchestra.run.smithi110:> mktemp 2024-09-13T05:05:13.375 INFO:teuthology.orchestra.run.smithi110.stdout:/tmp/tmp.37X02HJmT5 2024-09-13T05:05:13.375 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.37X02HJmT5 2024-09-13T05:05:13.554 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0666 /tmp/tmp.37X02HJmT5 2024-09-13T05:05:13.637 DEBUG:teuthology.orchestra.remote:smithi110:/tmp/tmp.37X02HJmT5 is 471KB 2024-09-13T05:05:13.735 DEBUG:teuthology.orchestra.run.smithi110:> rm -fr /tmp/tmp.37X02HJmT5 2024-09-13T05:05:13.750 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-13T05:05:13.750 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T05:05:13.753 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-13T05:05:13.814 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-13T05:05:13.814 DEBUG:teuthology.orchestra.run.smithi053:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T05:05:13.821 DEBUG:teuthology.orchestra.run.smithi110:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-13T05:05:13.847 INFO:teuthology.orchestra.run.smithi053.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T05:05:13.880 INFO:teuthology.orchestra.run.smithi110.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-13T05:05:13.885 INFO:tasks.ceph:Archiving crash dumps... 2024-09-13T05:05:13.886 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/remote/smithi053/crash 2024-09-13T05:05:13.887 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T05:05:13.919 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/remote/smithi110/crash 2024-09-13T05:05:13.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-13T05:05:13.949 INFO:tasks.ceph:Compressing logs... 2024-09-13T05:05:13.949 DEBUG:teuthology.orchestra.run.smithi053:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:05:13.964 DEBUG:teuthology.orchestra.run.smithi110:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:05:22.545 INFO:tasks.ceph:Archiving logs... 2024-09-13T05:05:22.546 DEBUG:teuthology.misc:Transferring archived files from smithi053:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/remote/smithi053/log 2024-09-13T05:05:22.547 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T05:05:23.585 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/remote/smithi110/log 2024-09-13T05:05:23.586 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-13T05:05:24.490 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-13T05:05:24.499 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-13T05:05:24.499 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T05:05:24.501 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-13T05:05:24.596 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-13T05:05:24.607 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-13T05:05:24.608 DEBUG:teuthology.orchestra.run.smithi053:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T05:05:24.610 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-13T05:05:24.626 INFO:teuthology.orchestra.run.smithi053.stderr:bash: line 1: ntpq: command not found 2024-09-13T05:05:24.626 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-09-13T05:05:24.631 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T05:05:24.631 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-09-13T05:05:24.631 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 60 +299us[ +262us] +/- 66ms 2024-09-13T05:05:24.631 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -534us[ -534us] +/- 52ms 2024-09-13T05:05:24.631 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv03.front.sepia.ceph.com 3 6 377 59 +385us[ +348us] +/- 66ms 2024-09-13T05:05:24.631 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T05:05:24.632 INFO:teuthology.orchestra.run.smithi053.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-13T05:05:24.633 INFO:teuthology.orchestra.run.smithi053.stdout:=============================================================================== 2024-09-13T05:05:24.633 INFO:teuthology.orchestra.run.smithi053.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +284us[ +278us] +/- 66ms 2024-09-13T05:05:24.633 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -555us[ -561us] +/- 52ms 2024-09-13T05:05:24.633 INFO:teuthology.orchestra.run.smithi053.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 59 +374us[ +368us] +/- 66ms 2024-09-13T05:05:24.633 INFO:teuthology.orchestra.run.smithi053.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-13T05:05:24.633 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-13T05:05:24.641 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-13T05:05:24.642 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-13T05:05:24.650 DEBUG:teuthology.orchestra.run.smithi053:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T05:05:24.754 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-13T05:05:24.828 DEBUG:teuthology.orchestra.run.smithi053:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T05:05:24.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T05:05:24.860 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-13T05:05:24.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T05:05:24.889 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-13T05:05:24.899 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-13T05:05:24.908 INFO:teuthology.task.internal:Duration was 616.284747 seconds 2024-09-13T05:05:24.908 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-13T05:05:24.917 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-13T05:05:24.917 DEBUG:teuthology.orchestra.run.smithi053:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T05:05:24.920 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-13T05:05:24.960 INFO:teuthology.orchestra.run.smithi053.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T05:05:24.971 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-13T05:05:25.286 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-13T05:05:25.286 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi053.front.sepia.ceph.com 2024-09-13T05:05:25.287 DEBUG:teuthology.orchestra.run.smithi053:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T05:05:25.349 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-09-13T05:05:25.350 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-13T05:05:25.376 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-13T05:05:25.376 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:05:25.392 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-13T05:05:25.444 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-13T05:05:25.444 DEBUG:teuthology.orchestra.run.smithi053:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T05:05:25.460 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-13T05:05:25.597 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-13T05:05:25.607 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-13T05:05:25.607 DEBUG:teuthology.orchestra.run.smithi053:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T05:05:25.633 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-13T05:05:25.658 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-13T05:05:25.667 DEBUG:teuthology.orchestra.run.smithi053:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:05:25.675 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:05:25.697 INFO:teuthology.orchestra.run.smithi053.stdout:kernel.core_pattern = core 2024-09-13T05:05:25.721 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-09-13T05:05:25.770 DEBUG:teuthology.orchestra.run.smithi053:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:05:25.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T05:05:25.785 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-13T05:05:25.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-13T05:05:25.825 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-13T05:05:25.835 INFO:teuthology.task.internal:Transferring archived files... 2024-09-13T05:05:26.061 DEBUG:teuthology.misc:Transferring archived files from smithi053:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/remote/smithi053 2024-09-13T05:05:26.062 DEBUG:teuthology.orchestra.run.smithi053:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T05:05:26.104 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-10_22:16:02-rbd-reef-distro-default-smithi/7900839/remote/smithi110 2024-09-13T05:05:26.104 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-13T05:05:26.137 INFO:teuthology.task.internal:Removing archive directory... 2024-09-13T05:05:26.137 DEBUG:teuthology.orchestra.run.smithi053:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T05:05:26.140 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-13T05:05:26.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-13T05:05:26.196 INFO:teuthology.task.internal:Not uploading archives. 2024-09-13T05:05:26.196 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-13T05:05:26.206 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-13T05:05:26.206 DEBUG:teuthology.orchestra.run.smithi053:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T05:05:26.208 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-13T05:05:26.222 INFO:teuthology.orchestra.run.smithi053.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 05:05 /home/ubuntu/cephtest 2024-09-13T05:05:26.242 INFO:teuthology.orchestra.run.smithi110.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 13 05:05 /home/ubuntu/cephtest 2024-09-13T05:05:26.243 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-13T05:05:26.253 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-13T05:05:26.274 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_cache_writethrough} duration: 616.2847468852997 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-13T05:05:26.274 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-13T05:05:26.388 INFO:teuthology.run:pass